HHS Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated to the Public

The U.S. Department of Health and Human Services (HHS) developed these guidelines to implement Office of Management and Budget (OMB) January 2002 requirement that all federal agencies issue guidelines for ensuring the quality of the information that they disseminate to the public. The Department of Health and Human Services is the United States government's principal agency for protecting the health of all Americans and providing essential human services, especially to those who are least able to help themselves. The Department includes more than 300 programs, covering a wide spectrum of activities.

In the course of carrying out program missions, agencies and staff offices within HHS disseminate a wide variety of information to the public, ranging from research and statistical reports to authoritative health and medical information. Many of these information dissemination activities and products rank among the highest quality scientific, statistical and programmatic information among federal agencies, and in many cases set the national and international standard for quality.

HHS is committed to disseminating information that meets the standards of quality set forth in OMB and in the guidelines discussed in this document. It is HHS's goal to ensure and maximize the quality, objectivity, utility, and integrity of information that it disseminates to the public. We strive to provide information that is accurate, reliable, clear, complete, unbiased, and useful. We are committed to integrating the principle of information quality into every phase of information development, including creation, collection, maintenance, and dissemination. The guidelines that follow describe the quality assurance policies and practices that support information dissemination activities in HHS.

Following an overview of the OMB Guidelines and HHS guidelines, the quality assurance policies of each of the major operating agencies and staff offices of HHS are described. Each set of agency guidelines includes a description of a) the mission of the agency, b) the scope and applicability of the guidelines within the agency, c) the types of information that the agency disseminates, d) the dissemination methods employed by the agency, e) the policies, standards and practices that the agency employs to ensure the quality of the information it disseminates, and f) an administrative mechanism and contact points for each agency so that individuals may seek correction of any information that is believed not to meet the OMB, HHS, or agency-specific guidelines along with an administrative appeals process.

On September 28, 2001, and as amended on February 22, 2002, OMB issued final Guidelines to implement section 515 of the Treasury and General Government Appropriations Act for Fiscal Year 2001 (Public Law 106-554). The statute directs OMB to "issue government wide guidelines that provide policy and procedural guidance to federal agencies for ensuring and maximizing the quality, objectivity, utility, and integrity of information (including statistical information) disseminated by federal agencies." By October 1, 2002, agencies must issue their own implementing guidelines. The guidelines only apply to information that is disseminated on or after October 1, 2002. The administrative mechanism for correction applies to information that the agency disseminates on or after October 1, 2002, regardless of when the agency first disseminated the information.

In general, the OMB Guidelines require agencies to adopt a basic standard of quality as a performance goal and take appropriate steps to incorporate information quality criteria into agency information dissemination practices. Quality is to be ensured and established at levels appropriate to the nature and timeliness of the information to be disseminated, and specific standards may be adopted that are appropriate to the various categories of information that is disseminated. Agencies are to develop a process for reviewing the quality of information before it is disseminated. Further, information quality is to be treated as an integral step in every aspect of the information development process.

In issuing the Guidelines, OMB outlined several guiding principles. First, OMB designed the Guidelines to apply to a wide variety of government dissemination activities that may range in importance and scope. OMB also designed the Guidelines to be generic enough to fit all media, whether printed or electronic. OMB specifically sought to avoid the problems inherent in developing detailed, prescriptive, "one size fits all" guidelines that would artificially require all types of dissemination activities to be treated in the same manner. Second, OMB designed the Guidelines so that agencies will meet basic information quality standards. The Guidelines recognize that some government information may need to meet higher or more specific standards than others, depending on their purpose and scope. The more important the information, the higher the quality standards to which it might be held, for example, "influential scientific, financial or statistical information" described below. At the same time, OMB recognizes that information quality comes at a cost. Accordingly, agencies are encouraged to weigh the costs and benefits of higher information quality in the development of information, and the level of quality to which it will be held.

Third, OMB designed the Guidelines so that agencies can apply them in a common sense, workable manner. OMB expects agencies to use existing processes rather than create new and potentially duplicative or contradictory processes. Finally, OMB recognizes that the Guidelines cannot be implemented in the same way by all agencies. While the implementation may differ, the essence of the Guidelines will apply. The agencies must make their methods transparent by providing documentation, ensure quality by reviewing the underlying methods used, by consulting as needed with both experts and users, and by keeping users notified about corrections and revisions. These underlying principles apply equally well across the diversity of HHS agencies and information dissemination activities, and they have been adopted in the approach to the HHS Guidelines described below.

In accordance with the OMB Guidelines, agencies subject to the Paperwork Reduction Act (44USC 3502(1)) are required to:

• Issue their own information quality guidelines ensuring and maximizing the quality, objectivity, utility, and integrity of information (including statistical information) disseminated by the agency not later than one year after the issuance of the OMB Guidelines;
• Establish administrative mechanisms allowing affected persons to seek and obtain correction of information maintained and disseminated by the agency that does not comply with the guidelines. Agencies also are to specify appropriate time periods for agency decisions on whether and how to correct the information, and are to notify the affected persons of the action taken. If the person who requested the correction does not agree with the agency's decision (including the corrective action, if any), that person may file for reconsideration within the agency. The agency is to establish an administrative appeal process to review the initial decision, and specify appropriate time limits in which to resolve such requests for reconsideration.
• Report periodically to the OMB Director on the number and nature of complaints received by the agency regarding the accuracy of information disseminated by the agency and how such complaints were resolved by the agency.

The HHS guidelines described in this plan incorporate the underlying principles that OMB used in designing their government-wide guidelines. First, the HHS guidelines apply to a wide range of government information dissemination activities across HHS and are generic enough to fit all types of media, including print, electronic, and other forms within HHS. Second, the HHS guidelines are intended to assure that all the information that is disseminated meets a basic level of quality and that more important information meets a more rigorous quality standard. Third, the HHS guidelines explicitly recognize the very different types of information that various HHS agencies disseminate depending on their missions, including the need for flexibility in implementation and avoidance of a "one size fits all" approach. Fourth, the statement of HHS information quality policies and procedures are issued in the form of guidelines and not a regulation.

HHS itself encompasses a broad and diverse range of health and human services programs which, while unified in their pursuit of broad goals, are themselves very diverse, encompassing the nation's largest health insurance plan, the nation's preeminent biomedical research agency, as well as most of the nation's federal capacity for public health protection and preparedness and income assistance to needy families. Accordingly, the HHS approach to implementation of the OMB Guidelines is designed to allow HHS agencies and offices to use existing agency quality assurance mechanisms, and apply the guidelines in a flexible manner that recognizes the mission of the agency, the wide range of data that is disseminated and the frequent reliance on third party sources.

1. Purpose

These Guidelines describe the policies and procedures that HHS agencies employ to ensure the quality of the information they disseminate and the administrative complaint mechanisms that HHS agencies make available to the public. The Guidelines provide policy and procedural guidance to HHS staff and are intended to inform the public about agency quality assurance policies and procedures. HHS views the guidelines as an evolving document and process. HHS will continually review the performance of the guidelines in the context of agency statutes and missions and will make revisions to the guidelines as necessary.

2. Definitions

1. "Quality" is an encompassing term comprising utility, objectivity, and Integrity. Therefore, the Guidelines sometimes refer to these four statutory terms, collectively, as "quality."
2. "Utility" refers to the usefulness of the information to its intended users, including the public. In assessing the usefulness of information that the agency disseminates to the public, the agency needs to consider the uses of the information not only from the perspective of the agency but also from the perspective of the public. As a result, when transparency of information is relevant for assessing the information's usefulness from the public's perspective, the agency must take care to ensure that transparency has been addressed in its review of the information.
3. "Objectivity" involves two distinct elements, presentation and substance. "Objectivity" includes whether disseminated information is being presented in an accurate, clear, complete, and unbiased manner. This involves whether the information is presented within a proper context. Sometimes, in disseminating certain types of information to the public, other information must also be disseminated in order to ensure an accurate, clear, complete, and unbiased presentation. Also, the agency needs to identify the sources of the disseminated information (to the extent possible, consistent with confidentiality protections) and, in a scientific, financial, or statistical context, the supporting data and models, so that the public can assess for itself whether there may be some reason to question the objectivity of the sources. Where appropriate, data should have full, accurate, transparent documentation, and error sources affecting data quality should be identified and disclosed to users. In addition, "objectivity" involves a focus on ensuring accurate, reliable, and unbiased information. In a scientific, financial or statistical context, the original and supporting data shall be generated, and the analytic results shall be developed, using sound statistical and research methods.
   1. If data and analytic results have been subjected to formal, independent, external peer review, the information may generally be presumed to be of acceptable objectivity. However, this presumption is rebuttable based on a persuasive showing by the petitioner in a particular instance. If agency-sponsored peer review is employed to help satisfy the objectivity standard, the review process employed shall meet the general criteria for competent and credible peer review recommended by OMB-OIRA to the President's Management Council (9/20/01) (www.whitehouse.gov/omb/inforeg/oira_review-process.html), namely, that (a) peer reviewers be selected primarily on the basis of necessary technical expertise, (b) peer reviewers be expected to disclose to agencies prior technical/policy positions they may have taken on the issues at hand, (c) peer reviewers be expected to disclose agencies their sources of personal and institutional funding (private or public sector), and (d) peer reviews be conducted in an open and rigorous manner.
   2. If an agency is responsible for disseminating influential scientific, financial, or statistical information, agency guidelines shall include a high degree of transparency about the data and methods to facilitate the reproducibility of such information by qualified third parties. Original and supporting data must be subject to commonly accepted scientific, financial, and statistical standards related thereto. However, agency guidelines shall not require that all disseminated data be subjected to a reproducibility requirement. Agencies may identify, in consultation with the relevant scientific and technical communities, those particular types of data that can practicably be subjected to a reproducibility requirement, given ethical, feasibility, or confidentiality constraints. It is understood that reproducibility of data is an indication of transparency about research design and methods and thus a replication exercise (i.e., a new experiment, test, or sample) shall not be required prior to each dissemination. With regard to analytic results, agency guidelines shall generally require sufficient transparency about data and methods that an independent reanalysis could be undertaken by a qualified member of the public. These transparency standards apply to agency analysis of data from a single study as well as to analyses that combine information from multiple studies. Making the data and methods publicly available will assist in determining whether analytic results are reproducible. However, the objectivity standard does not override other compelling interests such as privacy, trade secrets, intellectual property, and other confidentiality protections. In situations where public access to data and methods will not occur due to other compelling interests, agencies shall apply especially rigorous robustness checks to analytic results and document what checks were undertaken. However, agency guidelines, in all cases, shall require a disclosure of the specific data sources that have been used and the specific quantitative methods and assumptions that have been employed. Each agency is authorized to define the type of robustness checks, and level of detail for documentation thereof, in ways appropriate for it given the nature and multiplicity of issues for which the agency is responsible. With regard to analysis of risks to human health, safety and the environment maintained or disseminated by the agencies, agencies shall either adopt or adapt the quality principles applied by Congress to risk information used and disseminated pursuant to the Safe Drinking Water Act Amendments of 1996 (42 U.S.C. 300g-1(b)(3)(A) & (B)). Agencies responsible for dissemination of vital health and medical information shall interpret the reproducibility and peer-review standards in a manner appropriate to assuring the timely flow of vital information from agencies to medical providers, patients, health agencies, and the public. Information quality standards may be waived temporarily by agencies under urgent situations (e.g., imminent threats to public health or homeland security) in accordance with the latitude specified in agency-specific guidelines.
4. "Integrity" refers to the security of information — protection of the information from unauthorized access or revision, to ensure that the information is not compromised through corruption or falsification.
5. "Information" means any communication or representation of knowledge such as facts or data, in any medium or form, including textual, numerical, graphic, cartographic, narrative, or audiovisual forms. This definition includes information that an agency disseminates from a web page, but does not include the provision of hyperlinks to information that others disseminate. This definition does not include opinions, where the agency's presentation makes it clear that what is being offered is someone's opinion rather than fact or the agency's views.
6. "Government information" means information created, collected, processed, disseminated, or disposed of by or for the Federal Government.
7. "Information dissemination product" means any book, paper, map, machine-readable material, audiovisual production, or other documentary material, regardless of physical form or characteristic, an agency disseminates to the public. This definition includes any electronic document, CD-ROM, or web page.
8. "Dissemination" means agency initiated or sponsored distribution of information to the public (see 5 C.F.R. 1320.3(d) (definition of "Conduct or Sponsor"). Dissemination does not include distribution intended to be limited to government employees or agency contractors or grantees; intra- or inter-agency use or sharing of government information; and responses to request for agency records under the Freedom of Information Act, the Privacy Act, the Federal Advisory Committee Act or other similar law. This definition also does not include distribution limited to correspondence with individuals or persons, archival records, public filings, subpoenas or adjudicative processes.
9. "Influential," when used in the phrase "influential scientific, financial, or statistical information," means that the agency can reasonably determine that dissemination of the information will have or does have a clear and substantial impact on important public policies or important private sector decisions. Each agency is authorized to define "influential" in ways appropriate for it given the nature and multiplicity of issues for which the agency is responsible.
10. "Reproducibility" means that the information is capable of being substantially reproduced, subject to an acceptable degree of imprecision. For information judged to have more (less) important impacts, the degree of imprecision that is tolerated is reduced (increased). If agencies apply the reproducibility test to specific types of original or supporting data, the associated guidelines shall provide relevant definitions of reproducibility (e.g., standards for replication of laboratory data). With respect to analytic results, "capable of being substantially reproduced" means that independent analysis of the original or supporting data using identical methods would generate similar analytic results, subject to an acceptable degree of imprecision or error.

3. Scope and Applicability of the Guidelines

The HHS guidelines described in this implementation plan apply to substantive information dissemination activities that are initiated or sponsored by HHS agencies. The pre-dissemination review described in the guidelines only applies to information that is disseminated on or after October 1, 2002. The administrative mechanism for correction applies to information that the agency disseminates on or after October 1, 2002, regardless of when the agency first disseminated the information. The guidelines do not apply to the large proportion of extramural scientific research activity supported by HHS whose dissemination is the sole responsibility of the academic researcher rather than HHS. The guidelines do apply to the dissemination of information by federal intramural researchers if the dissemination is agency-initiated or sponsored. Otherwise, appropriate disclaimers are to be included in the report or speech to distinguish the research from agency views and positions. The guidelines do not apply to distribution of information limited to correspondence with individuals or persons, press releases that support the announcement or give public notice of information that the agency disseminates elsewhere, archival material, public filings, subpoenas or adjudicative processes. Nor do they apply to opinions, when the agency's presentation makes clear that what is disseminated is someone's opinion rather than fact or agency views.

4. Overview of HHS Information Dissemination and Quality Assurance

a. The HHS Mission

The Department of Health and Human Services is the United States government's principal agency for protecting the health of all Americans and providing essential human services, especially to those who are least able to help themselves. The Department includes more than 300 programs, covering a wide spectrum of activities. Program responsibilities include:

• Ensuring a safe and healthy America
• Conducting and supporting medical and social science research
• Preventing outbreaks of infectious diseases
• Assuring the safety of foods and drugs
• Administering the Medicare and Medicaid programs
• Providing financial assistance for low income families
• Improving maternal and child health
• Administering the Head Start program
• Preventing child abuse and domestic violence
• Providing assistance to States for substance abuse treatment and prevention
• Providing services for older Americans
• Assuring comprehensive health services for Native Americans

b. Categories of Information Disseminated

The development and dissemination of timely and high quality data and information is a critical component of the missions of many HHS programs, as well as those of HHS partners in the health and human services communities. As a result, HHS plays a major role in information dissemination, as a producer and user of high quality data and information, as a collaborator with partners in the health and human services communities, and as a national leader in health and human services information policy. In carrying out their diverse statutes and missions, HHS agencies develop and support the dissemination of the following types of substantive information:

• Scientific research studies (the results of biomedical, behavioral, services research and social science research)
• Statistical and analytic studies (the results of surveys, statistical systems as well as analytical and modeling studies and public use data files
• Programmatic, administrative and regulatory information, including program evaluations
• Authoritative health, medical and human services information aimed at consumers and health and human services professionals
• Public health surveillance, epidemiology and risk assessment studies and information

c. Types of Dissemination Mechanisms

HHS agencies disseminate information through a wide range of methods and print and electronic media. These include publication in peer reviewed literature, published reports, periodicals, newsletters, brochures, clearinghouses, websites, CD-ROM and other electronic media.

d. Overview of Quality Assurance Policies and Practices in HHS

Depending upon their specific statutes and missions and the nature of the information they disseminate to the public, HHS agencies currently use a variety of quality assurance methods and procedures. These methods and procedures are designed to maximize the quality of HHS information, including the objectivity, utility, and integrity.

• Objectivity involves a focus on ensuring that information is accurate, reliable and unbiased and that information products are presented in an accurate, clear, complete and unbiased manner. Objectivity is achieved by using reliable data sources and sound analytical techniques, and carefully reviewing information products prepared by qualified people using proven methods.
• Utility involves the usefulness of the information to its intended users. Utility is achieved by staying informed of information needs and developing new data, and information products where appropriate. Based on internal analyses of information requirements, convening and attending conferences, working with advisory committees and stakeholders, sponsoring outreach activities, and where appropriate, testing publications with targeted audiences to ensure relevance, clarity, and comprehensiveness, HHS agencies keeps abreast of information needs.
• Integrity refers to the security of information from unauthorized access or revision to ensure that the information is not compromised through corruption or falsification. HHS agencies have in place rigorous controls to ensure the integrity of its administrative information. Three distinct objectives are pursued in protecting the integrity of information: ensuring that there is no deliberate or accidental improper disclosure of sensitive automated information; protecting against deliberate or accidental corruption of automated information; and protecting against deliberate or accidental actions that cause automated information resources to be unavailable to users when needed. Information is accorded protection against disclosure, alteration, loss, or destruction based on the degree of sensitivity. In addition, HHS agencies use appropriate safeguards to protect data from improper disclosure by backing up critical data periodically, and, if a security incident occurs, by following proper incident response procedures. Managers are responsible for ensuring that employees, both Government and contractors, observe all security requirements, and that employees receive appropriate security training. HHS also is instituting an enterprise-wide structured management control review process that applies throughout the system life cycle. As part of this process, risk analyses are conducted to establish a balance between an acceptable level of risk and the costs associated with safeguards.

In addition, HHS is subject to a number of statutory requirements that protect the sensitive information it gathers and maintains on individuals. Among these are:

• Health Insurance Portability and Accountability Act of 1996
• Privacy Act of 1974
• Computer Security Act of 1987
• Office of Management and Budget (OMB) Circulars A-123, A-127, and A-130
• Government Information Security Reform Act
• Federal Managers' Financial Integrity Act (FMFIA) of 1982

e. HHS Information Quality Goals

The development of data and information within HHS is generally undertaken within the context of two overarching goals:

• attention to information quality as a total and continuing process, and
• commitment to making data and information supported with public funds available to the public, consistent with confidentiality concerns and resource availability. Further, when HHS agencies prepare a Paperwork Reduction Act (PRA) clearance submission, they strive to engage in a data development effort that will result in information that will be collected maintained, and used in a way that is consistent with OMB, HHS, and agency-specific information quality guidelines. As a general policy, HHS views data and information quality as a continuing process that begins at the inception of the information development process with project conceptualization and carries through all phases of data planning, design and execution, including information dissemination activities. Further, HHS agencies that support or sponsor research and statistical activities are encouraged to not only describe the methods and data sources in a clear and transparent manner in the reports they release, but also to make the data used for the report available to the public through public use data files, restricted access research files, research data centers, data archives and other mechanisms consistent with confidentiality, legal and proprietary restrictions. In addition, HHS agencies review the quality (including the objectivity, utility, and integrity) of information before it is disseminated and treat information quality as integral to every step of the development of information, including its creation, collection, maintenance and dissemination. The quality assurance processes that are used to ensure the quality of specific categories of information are described below.
  • Research and scientific studies disseminated by HHS are subject to an external, objective peer review process at both the inception stage and the pre-dissemination stage as a part of the publication process in peer reviewed journals. In addition, the quality of all of the intramural research programs of HHS agencies is continually reviewed and monitored by advisory committees and boards of scientific advisors. In accordance with widely accepted scientific research practice in the U.S., research reports disseminated by HHS agencies describe the methods, data sources, analytical techniques, measures, assumptions and limitations of the research, so that the study could be substantially reproduced. If original data are employed, it is the policy of HHS to make every effort to make the data available to the public in de-identified form consistent with confidentiality requirements, proprietary restrictions and resource availability.
  • Statistical activities of HHS agencies are based on reliable data sources and are carried out in accordance with modern statistical theory and practice, including scientific sampling, statistical inference and analytical techniques and practices. All statistical programs employ or have access to experts in statistics and research design. HHS houses the National Center for Health Statistics, the federal government's designated general purpose statistical agency for health statistics, as well as programmatic and special purpose statistical activities. All proposals for original data collection activities in HHS undergo a rigorous and exacting review process in connection with the Paperwork Reduction Act, which also provides opportunity for public comment in the design of the information collection. Frequent meetings with user groups are common, and individual surveys and statistical systems often employ project specific technical advisory groups. Substantive reports from HHS statistical activities undergo a quality review process within their organizations before they are released, including expert review by supervisors, internal peer review by qualified scientists and statisticians, and in some cases external peer review as well as expert review by other offices prior to dissemination.
  • Programmatic and administrative information — A significant amount of substantive information is disseminated by HHS agencies in connection with and as a byproduct of the administration of programs. Often the programmatic and administrative information disseminated is obtained from third parties, such as States, grantees, health plan contractors or intermediaries, or community-based organizations. In their stewardship function, agencies often collect, compile, standardize, analyze and disseminate such programmatic information. While the reliance on third parties places limits on the federal quality assurance authorities, a variety of techniques are employed to promote the accuracy, completeness and timeliness of the information. These include use of generally accepted accounting and financial management procedures and principles, internal controls, legal certifications and assurances on the part of the organizations supplying the information, audited financial reports and statements, as well as sample audits and site visits, and checks for completeness and consistency with trends and external controls. Programmatic reports are typically subject to supervisory review before release.
  • Program evaluation studies are often undertaken by HHS agencies to assess program functioning and identify opportunities for improvement. Agencies employ quality assurance procedures in the choice and development of evaluation projects. Proposals for evaluation activities in HHS agencies are usually reviewed by agency management for suitability, utility and methodology in accordance with sound evaluation design and standards of evaluation practice. Many evaluation projects have specific technical advisory committees that oversee the design and conduct of the evaluation in accordance with standard evaluation theory and practice, and they often provide an expert review of the final report. Results of evaluation activities are released to the public only after agency management has completed a review of the quality, accuracy and completeness of the report.
  • Regulatory information — A variety of information is used in support of regulatory development and decision-making. Regulatory activity undertaken by HHS agencies closely tracks statutory authorities and program responsibilities. Scientific, financial, and statistical information used in support of regulatory decision-making is subject to a quality review process within the agency involving appropriate experts depending upon the nature of the information. In addition, all significant proposed regulations are reviewed by OMB prior to issuance for public comment, and all proposed regulatory actions provide for extensive public comment.
  • Authoritative health, scientific and consumer information — Several HHS agencies develop and disseminate authoritative health and human services information intended for consumers and the professional community. In some instances, the agency simply provides a link to information developed by other authoritative organizations. In other instances, the agency develops its own consumer and professional practice information. In the latter case, the information is reviewed for scientific and medical accuracy and completeness by experts within the agency before it is disseminated. In a number of instances the information also is reviewed by scientific and medical advisory bodies before dissemination as well, depending on the nature of the information.
  • Public health surveillance and epidemiological information — Several HHS agencies compile, analyze and disseminate information from public health surveillance systems and epidemiological activities. In many surveillance systems, the primary information is developed by State and local government agencies, clinical laboratories and other health care entities and reported to CDC for national aggregation and analysis. Data quality is assured through use of reliable data sources, appropriate statistical techniques, agreement on national reporting standards, quality control procedures, standard case definitions and reports, adherence to professional practices and standards for public health reporting in the U.S., and frequent consultation with the user community. Before such information is disseminated, it is reviewed for medical, scientific and public health accuracy, soundness and utility by agency experts. Comment and feedback is encouraged on such information, and HHS agencies work closely with the relevant professional and public health organizations.

f. Influential Scientific, Financial, and Statistical Information

From time to time, HHS agencies disseminate influential scientific, financial and statistical information as defined in the OMB Guidelines. In such instances, care is taken to ensure that the information is substantially reproducible and replicable. This goal is accomplished by using reliable data sources and sound analytical techniques, and by employing a high degree of transparency about the data, sources, methods, measures, assumptions and limitations used to develop the information in order to facilitate reproducibility by qualified third parties. In the case of original or supporting data, most major epidemiological and statistical activities sponsored by HHS agencies have well developed public use data dissemination programs that make much of the data available to the public in standardized, de-identified micro-data files. Because of confidentiality, ethical and feasibility constraints and legal obligations to third parties supplying the information, there may be instances where original or supporting data may not be available to the public, but HHS agencies typically will work with qualified third parties to facilitate understanding, and transparency in data sources and methods will be emphasized in the report or in reference documents. In the case of analytical studies, HHS agencies will make provisions for sufficient transparency about data and methods so that an independent reanalysis could be undertaken by a qualified member of the public.

g. Health, Safety, and Environmental Information

Several HHS agencies have science-based missions and use such information in decision making. These agencies adapt the quality standards discussed in the Safe Drinking Water Act Amendments of 1996 (SDWA) (42 U.S.C. 300g-1(b)(3)(A) and (B)). The adaptation involves the commitment of these agencies to use: a.) the best available science and supporting studies conducted in accordance with sound and objective scientific practices, including peer reviewed studies when available and b.) data collected by accepted methods (if reliability of the method and the nature of the decision justify use of the data.) Additional detail relating to the standards employed by individual agencies is described in Part II, the individual agency section of these guidelines.

h. Urgent Public Health and Safety Information

Several HHS agencies are responsible for dissemination of authoritative health, medical and safety information on a real time basis in order to protect the health of the public against urgent and emerging threats. Accordingly, nothing in these guidelines relating to reproducibility or peer review shall be construed to limit or delay the timely flow of vital information from agencies to medical providers, patients, health agencies, and the public. HHS reserves the right to waive information quality standards temporarily for agencies addressing urgent situations (e.g., imminent threats to public health or homeland security) in accordance with the latitude described in both the OMB and agency specific guidelines.

The OMB Guidelines require all agencies to establish administrative mechanisms allowing affected persons to seek and obtain correction of information disseminated by the agency that does not comply with OMB, HHS or agency-specific guidelines. Agencies also are to specify appropriate time periods for agency decisions on whether and how to correct the information, and are to notify the affected persons of the action taken. If the person who requested the correction does not agree with the agency's decision (including the corrective action, if any), that person may file for reconsideration within the agency. The agency is to establish an administrative appeal process to review the initial decision, and specify appropriate time limits in which to resolve such requests for reconsideration. The agency need not respond substantively to information not covered by the guidelines or to frivolous or repetitive requests for correction.

Requests for correction of information quality pursuant to Section 515 are to be directed to the respective agency. A common format for complaints has been developed across HHS. The approach is described below. To accompany the actual implementation of the Guidelines in October 2002, HHS has created a department-wide website describing in user friendly terms the procedures and contact persons for submitting requests for corrections. The web site can be accessed here.

• Responsibility of the Complainant
  In general, to seek an information quality request for correction of information disseminated by any HHS agency, individuals should follow the procedures described below. Requests for correction that are specific and provide evidence to support the need for correction will enable the agency to provide a satisfactory response. Complainants should be aware that they bear the "burden of proof" with respect to the necessity for correction as well as with respect to the type of correction they seek. A complaint or request for correction of information must be in written hard copy or electronic form, be sent to the agency designated address by mail or electronic-mail (e-mail); and state that an information quality request for correction is being submitted. In terms of content, the complaint letter must contain:
  • a detailed description of the specific material that needs to be corrected including where the material is located, i.e. the publication title, date, and publication number, if any, or the website and web page address (url), or the speech title, presenter, date and place of delivery;
  • the specific reasons for believing the information does not comply with OMB, HHS, or agency-specific guidelines and is in error and supporting documentation, if any;
  • the specific recommendations for correcting the information;
  • a description of how the person submitting the complaint is affected by the information error; and
  • the name, mailing address, telephone number, e-mail address, and organizational affiliation, if any, of the individual making the complaint.
• Responsibility of the Agency
  Based on a review of the information provided, the agency will determine whether a correction is warranted and if, so what action to take. The agency will respond to the requestor by letter or e-mail. The agency's response will explain the findings of the review and the actions that the agency will take, if any. The response will consider the nature and timeliness of the information involved and such factors as the significance of the correction on the use of the information, the magnitude of the correction and the resource requirements for the correction. The response will describe how the complainant may request reconsideration. The agency will respond to all requests for correction within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.
• Appeals for Reconsideration
  If the individual submitting the complaint does not agree with the agency's decision (including the corrective action), the complainant may send a written hard copy or electronic request for reconsideration within 30 days of receipt of the agency's decision. The appeal shall state the reasons why the agency response is insufficient or inadequate. Complainants shall attach a copy of their original request and the agency response to it, clearly mark the appeal with the words, "Information Quality Appeal," and send the appeal to the specific agency appeals address. The agency official who handles the original complaint will not have responsibility for resolving the appeal. The agency will respond to all requests for appeals within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.
• Rulemakings and Other Public Comment Procedures
  Existing public comment procedures for rule-makings and other formal agency actions already provide well established procedural safeguards that allow affected persons to raise information quality issues on a timely basis. Accordingly, agencies will use these existing procedures to respond to information quality complaints that arise in this process. In cases where the agency disseminates a study, analysis, or other information prior to the final agency action or information product, requests for correction will be considered prior to the final agency action or information product in those cases where in the agency's judgment issuing an earlier response would not unduly delay issuance of the agency action or information product and the complainant has shown a reasonable likelihood of suffering actual harm from the agency's dissemination if the agency does not resolve the complaint prior to the final agency action or information product.

I. Agency Mission

The Administration for Children and Families (ACF) administers federal programs that promote the economic and social well-being of children, youth, families, and communities. ACF programs include Temporary Assistance for Needy Families (TANF), child support enforcement, child care, Head Start, child welfare, refugee resettlement, Native American/Tribal programs, and more. Services are generally delivered by state, county, city, and tribal governments, as well as public and private local organizations. ACF supports these organizations through funding, policy guidance, technical assistance, and information services.

II. Scope and Applicability of Guidelines

ACF is committed to ensuring that disseminated information meets the quality standards set forth in OMB, HHS, and ACF guidelines. It is ACF's policy to ensure and maximize the quality, objectivity, utility, and integrity of information disseminated to the public. ACF strives to provide information that is accurate, reliable, clear, complete, unbiased, and useful. ACF integrates the principle of information quality into every phase of information development, including creation, collection, maintenance, and dissemination.

The pre-dissemination review applies to information disseminated on or after October 1, 2002. The administrative mechanism for correction applies to all information the agency disseminates on or after October 1, 2002, regardless of when first disseminated.

ACF information subject to the Information Quality Guidelines includes:

• Statistical information
• Studies and summaries prepared for public dissemination to inform the public about the impact of ACF programs
• Studies and summaries prepared for use in formulating broad program policy

The following types of information are not subject to the Information Quality Guidelines:

• Archival information
• Studies, reports, documents, summaries, and articles not authored by ACF and not representing ACF's views, including materials authored and distributed by ACF grantees
• Information limited to government employees (intra- or interagency) or agency contractors or grantees
• Information pertaining to basic agency operations or management information produced primarily for internal use
• Procedural or policy manuals
• Responses to requests for agency records under the Freedom of Information Act (FOIA), the Privacy Act, the Federal Advisory Committee Act, or similar laws
• Information relating solely to correspondence with individuals or persons
• Press releases that announce or give public notice of information ACF has disseminated elsewhere

III. Types of Information Disseminated to the Public

Given the wide variety of programs administered by ACF and the numerous types of information generated, it is not possible to offer a comprehensive list of all dissemination activities. The following examples represent a cross-section of ACF information dissemination. Note that not all information described below is subject to the OMB Guidelines.

Program Information — Descriptions of ACF programs; budget and funding information; data on grantees; types of services supported; fact sheets; program highlights; frequently asked questions; and other descriptive information available through ACF websites and publications. 

Statistical Information — Data on populations served through ACF programs, such as: characteristics of families served by TANF; child support caseload information; number and characteristics of children who are victims of child maltreatment or who are in foster care or adopted from public child welfare systems; number of children served in Head Start and their teachers' qualifications; characteristics of children served by runaway and homeless youth programs; number of children receiving child care subsidies under the Child Care and Development Fund; refugee arrival and placement data; and other program-specific statistics published in annual reports, data dashboards, and data briefs. 

Research and Evaluation Reports — Social science research and evaluation related to ACF programs and policy, made available through program office websites and the Office of Planning, Research and Evaluation (OPRE). Examples include evaluations of early childhood programs, studies on child welfare outcomes, and TANF research. 

Reports to Congress — Annual and periodic reports submitted to Congress in compliance with legislative provisions, such as child support enforcement reports, child maltreatment statistics, Head Start program performance, and refugee resettlement data. These reports are made publicly available following submission to Congress. 

IV. Types of Dissemination Methods

Information is disseminated through digital and print materials. Dissemination methods include: posting information on the ACF website (acf.gov) and program-specific websites; targeted notifications to state officials, tribal leaders, grantees, and program administrators; making information available through clearinghouses and resource centers; presenting and distributing information at conferences and webinars; and electronically collecting and distributing aggregated and disaggregated program data.

Research and evaluations funded by ACF are also disseminated through means such as peer-reviewed journals and other academic publications; distributions to state and tribal program directors and targeted audiences; notifications to partner organizations and networks; and presentations at research conferences and technical assistance events.

V. Quality Assurance Policies, Standards, and Processes

At ACF, quality assurance begins at the inception of information development. ACF reviews the quality (including objectivity, utility, and integrity) of information before dissemination and treats information quality as integral to every step of development, including creation, collection, maintenance, and dissemination.

Review procedures vary among programs depending on program requirements and the type of information to be disseminated:

Statistical Information: Statistical information is gathered from state agencies, tribal entities, local grantees, and internal program management data sources. To assure quality, integrity, and consistency, ACF staff review and analyze data, subject data to validation procedures, and contact relevant parties such as states, tribes, or grantees to seek clarification when discrepancies arise. Reporting entities are provided opportunities to review and revise their information prior to publication or posting on websites. 

Surveys: Surveys sponsored by ACF use methodologies consistent with generally accepted professional standards for all aspects of survey development, including sample frame development, statistical design, questionnaire design and testing, data collection, sampling and coverage error analysis, nonresponse analysis, imputation of missing data, and calculation of weights and variance estimates. ACF surveys follow guidelines set forth in the Paperwork Reduction Act (PRA) and related regulations. ACF demonstrates in its PRA clearance packages that each information collection will result in information collected, maintained, and used in a manner consistent with OMB, HHS, and ACF information quality guidelines. Major research and evaluation studies typically consult with technical advisory groups composed of state and tribal officials, academic experts, practitioners, and federal experts. 

Analytic Reports and Policy Studies: Reports submitted by grantees and contractors are subject to internal reviews by multiple agency officials for quality, objectivity, and accuracy. Federal experts from within and outside the Department review reports with cross-cutting implications. Information in analytical reports and policy studies is based on estimates derived from reliable administrative data and external data sources. All data sources are identified. All analytic reports and policy studies are reviewed by technically qualified staff to ensure valid, complete, unbiased, objective, and relevant analysis. More technically complex reports are also reviewed by subject matter experts outside the originating organizational component.

Policy documents are cleared through ACF and Departmental clearance procedures, including review by the Office of the General Counsel to ensure proper legal interpretation, as well as review by other knowledgeable officials. Technical advisory groups may review and comment on reports prior to release. All ACF regulations and reports to Congress are subject to review and clearance under both ACF and Departmental clearance processes.

Policy for Correcting Errors: If an error is detected in ACF publications before distribution, it is corrected. If materials have already been distributed, ACF issues a notification to recipients or includes an errata sheet with subsequent publications. Errors in materials on the ACF website are corrected online promptly. 

VI. Agency Administrative Complaint Procedures

ACF has developed administrative mechanisms to allow affected persons to seek and obtain correction of disseminated information that does not comply with OMB, HHS, and ACF guidelines.

External complaints about disseminated information can be made in written correspondence. The Office of Planning, Research and Evaluation (OPRE) is designated to receive and resolve complaints regarding information that does not comply with the OMB, HHS, or ACF guidelines. The office address is:

Administration for Children and Families
Office of Planning, Research & Evaluation
330 C Street, S.W.,
Washington, D.C. 20201

Feedback and complaints may be sent electronically to InfoQuality@acf.hhs.gov.

Responsibility of the Complainant

To seek a correction of information disseminated by the agency, individuals should follow these procedures:

• A complaint or request for review and correction shall be in written electronic or hard copy form
• It shall be sent to the agency by email or mail
• It shall state that an information quality request for correction is being submitted

The complaint shall contain:

• A detailed description of the specific material needing correction, including where the material is located (publication title, date, publication number, website URL, or other identifying information)
• The specific reasons for believing the information does not comply with OMB, HHS, or ACF guidelines and is in error, with supporting documentation if available
• Specific recommendations for correcting the information
• A description of how the person submitting the complaint is affected by the information error
• The name, mailing address, telephone number, email address, and organizational affiliation (if any) of the individual making the complaint

Complainants should be aware that they bear the "burden of proof" with respect to the necessity for correction and the type of correction sought.

Responsibility of the Agency

Based on a review of the information provided, the agency will determine whether a correction is warranted and, if so, what action to take. The agency will respond to the requestor by email or letter. The response will explain the findings of the review and the actions the agency will take, if any. The response will consider the nature and timeliness of the information and factors such as the significance and magnitude of the correction on the use of the information. The response will describe how the complainant may request reconsideration. The agency will respond to all requests for correction within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required, indicate the reason why, and provide an estimated decision date.

Appeals

If the individual submitting the complaint does not agree with the agency's decision (including the corrective action, if any), the complainant may send electronic or hard copy request for reconsideration within 30 days of receipt of the agency's decision. The appeal shall state the reasons why the agency response is insufficient or inadequate. Complainants shall attach a copy of their original request and the agency response, clearly mark the appeal with the words "Information Quality Appeal," and send the appeal to the email or office address listed above. The agency official who resolved the original complaint will not have responsibility for the appeal. The agency will respond to all appeals within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required, indicate the reason why, and provide an estimated decision date.

VII. Influential Scientific, Financial, and Statistical Information

A specific piece or body of information is “influential” when it is the principal basis for a decision by a government or private sector decision maker. That is, if the same decision would be difficult to reach in the absence of a piece or body of information, or if the decision would lose its fundamental scientific, financial, or statistical underpinning without the information, then that information is “influential.” If the decision could be reached in the absence of that specific piece or body of information, then the information is not “influential.”

To be influential, information should have a clear and substantial impact. A clear impact is one that is determined by the agency to have a high probability of occurring. A substantial impact is one that meets the levels of importance described below.

In rulemaking, influential information is scientific, financial, or statistical information that the agency believes will have a clear and substantial impact on the resolution of one or more key issues in an economically significant rulemaking, as that term is defined in section 3(f)(1) of Executive Order 12866.

In non-rulemaking contexts, ACF should consider two factors in relation to each other - breadth and intensity - in determining whether information is influential. These factors need to be considered together. Information that has a low cost or modest impact on a limited range of affected parties is less likely to be influential than information that can have a very costly or crucial impact on a broad range of parties. Of course, even information that has a low cost or modest impact on any one party can be influential if it can impact a broad range of parties. It should be noted that the definition of "influential" applies to information itself and not to the decisions that the information may support. Even if a decision or action by an agency is very important, a particular piece of information supporting it may not be influential, for example, because it is cumulative to other information or because the decision involves legal or policy issues.

Moreover, if it is merely arguable that an impact will occur because of the specific information, or if it is a close judgment call, then the impact is probably not clear and substantial, and thus not “influential” for the purposes of applying the Guidelines.

Based on the above, ACF currently produces two pieces or bodies of “influential scientific, financial, or statistical information”: the Home Visiting Evidence of Effectiveness (HomVEE) Review and the Title IV-E Prevention Services Clearinghouse, which conduct objective and transparent reviews of the evidence of effectiveness for home visiting models that target families with pregnant women and children from birth to kindergarten entry, and programs and services intended to provide enhanced support to children and families and prevent foster care placements, respectively. These evidence reviews provide information about program models that meet HHS criteria for demonstrating evidence of effectiveness and are thus eligible for certain federal funding; state and potential grantees use this information to inform decisions about which program models to implement.

These evidence reviews assess the quality of multiple studies using detailed, systematic, pre-specified procedures and standards, which are made publicly available along with the results of the reviews. Studies included in the reviews are located by literature reviews and searches. In addition, the public is routinely invited to submit studies for review. Secondary review is built into the review procedures for quality assurance. Results are reported at the program level and include information about the level of confidence for results. The reviews are disseminated with detailed information about the review process, underlying studies reviewed, and review procedures and standards, sufficient to ensure replicability.

Stakeholders may request a re-review of the program or service rating based on the misapplication or criteria, missing information, or other errors. Requests for reconsideration can be made via an email request to the independent contractor conducting the review. The re-review follows standardized procedures, including assigning different, blinded reviewers to conduct any re-review. The requestor is notified of the results of the re-review, and any changes, if warranted, are made on the publication website.

I. Agency Mission

The mission of the Administration on Aging (AoA) is to ensure that older Americans have the opportunity to age with dignity, have choices in managing their own lives, and remain active and productive members of their families and communities. We fund states, tribal agencies, city and county governments, and non-profit organizations at the community level. Core programs include home and community-based services such as transportation, chore and personal care; home and community-based nutrition services; preventive health services; support to caregivers; services that protect the rights of older persons; services for Native Americans and persons with Alzheimer's Disease. AoA assists these organizations by providing policy direction, technical assistance and information. Results and accomplishments of AoA projects and activities are made available to the aging community and to the public at large.

II. Scope and Applicability of Guidelines

A. Covered Information 

AoA is committed to ensuring that disseminated information meets the standards of quality set forth in the OMB, HHS and AoA guidelines. It is AoA's goal to ensure and maximize the quality, objectivity, utility, and integrity of information that it disseminates to the public. We strive to provide information that is accurate, reliable, clear, complete, unbiased, and useful. We are committed to integrating the principle of information quality into every phase of information development, including creation, collection, maintenance, and dissemination.

The pre-dissemination review described in these guidelines only applies to information disseminated on or after October 1, 2002. The administrative mechanism for correction applies to information that the agency disseminates on or after October 1, 2002, regardless of when the agency first disseminated the information. The guidelines apply to the dissemination of substantive information that is endorsed or sponsored by the Agency. This includes:

1. Programmatic Analyses and Evaluations
2. Reports, Monographs, and Technical Assistance Materials, including brochures, documents, newsletters, electronic documents, and audiovisual productions, as well as reports and proceedings from AoA sponsored conferences and meetings
3. Consumer Information
4. Oral information, including speeches, interviews, expert opinions, only if representing AoA's views, official positions, or policies as well as written op-ed and similar editorial materials
    

B. Information Not Covered

1. CCCS bibliographic information or other archival records
2. Documents not authored by the agency and not representing the agency's views, including information authored and distributed by AOA grantees. Information that is intended to be limited in dissemination to Government employees or agency contractors or grantees
3. Information pertaining to basic agency operations, e.g., information about agency authorities, activities, programs, along with contact information for the public, organizational charts, AoA Status Reports, solicitations, program announcements (PAs), receipt and review materials (e.g., summary statements developed in connection with grant reviews)
4. Information intended solely for intra- or interagency use or sharing of Government information
5. Responses to requests for agency records under the Freedom of Information Act (FOIA), the Privacy Act, the Federal Advisory Committee Act, or other similar laws
6. Information relating solely to correspondence with individuals or persons
7. Press releases that support the announcement or give public notice of information that AoA has disseminated elsewhere.
8. Information intended to be limited to public filings, subpoenas, or adjudicative processes
9. Opinions where the agency's presentation makes it clear that what is being offered is personal opinion rather than fact or the agency's views
10. Information that is intended to be limited in dissemination to Government employees or agency contractors or grantees

III. Types of Information Disseminated by AoA to the Public

Each year, AoA produces a number of publications of various types, and about 500 static Web pages. All publications that carry the AoA imprimatur, i.e. are considered official AoA publications or releases and must follow AoA procedures for appropriate for that type of material (see Section V). The types of information disseminated by AoA to the public include the following:

1. Programmatic Analyses and Evaluations

   This category includes program reviews, evaluations and recurring reports. Analyses of project results may be developed and used for many purposes, including meeting annual reporting requirements, such as the Government Performance and Results Act (GPRA). Highlights of evaluations and analyses may be on the AoA Web site, and used in testimonies and speeches by AoA officials.

2. Reports, Monographs, and Technical Assistance Materials

   These are generally authored or co-authored by AoA staff analysts as part of their official duties or leading professionals in the field under contract. Such reports typically either provide technical assistance information to agencies and service providers working in aging, provide statistical compilations (typically of Census demographical data which is needed by aging agencies and they plan services) or the reports may address a program or policy issue of concern.

3. Consumer Information

   AoA provides a number of resources for the general consumer to learn about a variety of aging related concerns such as health information, housing programs available to older people and how to access the service system. A considerable amount of this information is distributed through AoA Center for Communications and Consumer Services (CCCS). Other sources of consumer information include distribution by other AoA staff and the AoA web site.

4. Oral information, including speeches, interviews

   AoA executives and experts make frequent speeches on a variety of issues relating to aging and are interviewed by the media (only if representing AoA's views, official positions, or policies) as well as written op-ed and similar editorial materials.

IV. Types of Dissemination Methods

AoA information is disseminated in many mediums, with the following being most common:

1. Print -- publications, books, newsletters, brochures, booklets, pamphlets, and reports.
2. Oral -- formal speeches, oral presentations, interviews, or commentaries for publication or broadcast; letters-to-the-editor or correspondence likely to result in similar publications.
3. Electronic -- The AoA Web site is a Central Information Resource in Aging. It is also a vehicle for enhancing the way AoA carries out its business operations. To accomplish this dual mission, AoA must put the needs of its users first. It serves as a dynamic source of credible, reliable and timely information and technical assistance designed to support our network, aging service providers, older people and their families in the daily conduct of their lives and work as well as helping the agency carry out its various business operations.

V. Quality Assurance Policies, Standards, and Processes for Ensuring the Quality of Information Disseminated to the Public

A. Overview

All AoA documents must be prepared in accordance with professional and ethical standards, as well as generally accepted standards of good taste. They must be appropriate for dissemination by this agency, and must undergo appropriate review and approval prior to release. At AoA, the quality assurance process begins at the inception of the information development process. AoA efforts to ensure and maximize information quality begin at the preparation stage, and continue through the review and approval stages. Existing AoA policies developed in concert with Federal computer security laws provide appropriate security safeguards to ensure integrity of AoA documents, i.e., ensure that the information is protected from unauthorized access, revision, corruption, or falsification.

AoA has quality control measures embedded in its grant application review process to ensure that the information ultimately disseminated is of high quality. AoA grant applications undergo extensive professional review.

B. AoA Information Review and Approval Policies and Procedures

Publications are expected to meet high standards of quality, make a substantial contribution to the field, and contain sufficient information for the informed audience to assess its validity. The review, approval, and dissemination of substantive information by AoA require adherence to appropriate clearance procedures for this type of material and are consistent with HHS guidelines. The originating office is responsible for obtaining the necessary clearances for reproduction and distribution of printed materials and should ensure that written material distributed is appropriate and consistent with HHS policy. A document that has obtained publication clearance for paper printing is often posted on the AoA web site for greater accessibility. AoA Web documents with no print counterpart require content clearance by the appropriate AoA office or contact person to ensure that the information observes all applicable requirements governing information for release to the public.

As noted above, AoA issues four types of information:

1. Programmatic Analyses and Evaluations
2. Reports, Monographs, and Technical Assistance Materials
3. Consumer Information
4. Oral Information

AoA reviews the quality (including the objectivity, utility, and integrity) of information before it is disseminated and treats information quality as integral to every step of the development of information, including its creation, collection, maintenance and dissemination. The information review and approval process is essentially the same for each of these types of information. Such information must meet the following guidelines which AoA employs to assure the quality of its information products, including their utility, objectivity, and validity:

Utility involves the usefulness of the information to its intended users. Utility is achieved by staying informed of information needs and developing and information products which are appropriate to these needs. This is achieved by: internal analyses of information requirements, continuous informal discussions and consultation with members of the aging services network, project grantees and other experts in the field, consultation with consumer groups, convening and attending conferences, working with professional groups, and sponsoring outreach activities. Where appropriate, contact information is available on each publication to allow feedback and questions by users.

Objectivity involves a focus on ensuring that information is accurate, reliable and unbiased and that information products are presented in an accurate, clear, complete and unbiased manner. Objectivity is achieved by using reliable data sources and sound analytical techniques, and preparing information products that use proven methods by qualified people that are carefully reviewed.

Integrity -- Information contained in analytical reports and technical assistance materials is based on valid and reliable data sources which are identified. All reports and materials are reviewed by technically qualified staff to ensure that analysis is valid, complete, unbiased, objective and relevant. Materials that are considered to be more technically complex are also reviewed by subject matter experts and other AoA program experts outside of the originating component to provide additional perspective and expertise.

C. Administrative Requirements

AoA printed publications must be cleared through the Assistant Secretary for Aging and the Office of the Assistant Secretary for Public Affairs (OASPA), HHS. Similarly, all AoA Audio/Visual projects and exhibits must be cleared through OASPA. Further, AoA will demonstrate in its PRA clearance packages that each draft information collection will result in information that will be collected, maintained, and used in a way that is consistent with OMB, HHS and AoA information quality guidelines.

In general, any writing by an AoA employee on a work-related subject, whether intended for electronic or print publication, or for oral delivery, must be prepared according to accepted AoA standards of quality, reviewed for substantive content, and administratively approved. The purpose of the AoA clearance process is to improve the quality of information, and to ensure the accuracy, objectivity, utility, and validity of information. Directors of AoA component units (or their delegates) are responsible for establishing and maintaining controls to ensure competent and timely clearance of professional writing and presentations by developing procedures appropriate to the type of information.

Oral information, including speeches, interviews, expert opinions, only if representing AoA views, official positions, or policies, including any statements, comments, or discussion of Federal policies or practices that are relevant to the employee's position or duties, draw conclusions, advocate or oppose professional practices or positions on subjects related to AoA duties, or might otherwise be construed as reflecting an official position by AoA, HHS, or the Federal Government, are covered by the OMB Guidelines, and must be approved in the Office of the Assistant Secretary for Aging or the Director of CCCS, AoA.

No review or approval is required for nonofficial and private writing, speaking, and publishing by an employee unless his/her AoA employment is likely to be regarded as influencing the content.

D. Special Considerations for Agency Dissemination

Disclaimers -- Normally, the need for a disclaimer is eliminated through the clearance process. However, a disclaimer may still be needed even after official clearance to clarify that the presentation should not be construed as necessarily representing AoA views, and/or to distinguish the status of information (e.g., preliminary, based on partial data set). The Department's regulations (Standards of Conduct) to which the AoA subscribes, require that disclaimers be used in all unofficial writing and editing related to the employee's official duties and/or affiliation with programs of the Federal Government in which the employee's identification with AoA is to be shown, can be inferred, or is well-known.

E. AoA Center for Communications and Consumer Services (CCCS)

The Center for Communications and Consumer Services (CCCS), a unit of AoA, is a central source for a wide variety of consumer as well as program- and policy-related materials and demographic and other statistical data on the health, economic, and social status of older Americans. CCCS services include the following: inquiry response, repository of AoA-funded research products, a bibliographic database of AoA-funded r&d products and other aging-related products, a reading room and reference collection, and statistical information.

At AoA, there are essentially three types of materials being disseminated through information services to the public: (1) Materials produced by AoA staff or contractors that undergo usual AoA review and approval processes; (2) materials produced by AoA grantees that are subject to policies and procedures in the AoA Grants Administration Manual; and (3) other materials not produced by AoA but available from a variety of professional sources (e.g., brochures handbooks, statistical tables, articles) whether in print or in electronic format, with appropriate disclaimers attached. Non-Federal materials typically undergo careful professional review before they are disseminated by AoA information staff. Such materials are accompanied by appropriate disclaimers.

F. Web Site Information

AoA has designated content area experts to develop or recommend material for inclusion on the AoA web site. These content area specialists periodically review material on the Web page to determine whether it is accurate and up to date. Information, particularly time-sensitive information, should be posted as soon as possible. Web page creators are expected to promptly update or remove out-of-date information.

Unless noted otherwise, information posted on pages within the "AoA.Gov" domain is considered to be "in the public domain." As such, others are free to establish links to AoA online resources. In establishing such links, AoA requests that others avoid creating the impression that AoA is endorsing or promoting any particular product or service. In the same vein, any outside link to an external resource from an AoA Web site is examined on a case-by-case basis. In general, the content manager for each page determines when links to outside entities are justified.

AoA web pages containing links to external web pages not located on AoA servers should include a link to a statement that releases AoA from responsibility for the material included in the external Web page. Again, it is important to avoid giving a user the impression that AoA is endorsing information or a service or product described in an external site.

VI. Agency Administrative Complaint Procedures

Responsible Official

AoA has developed administrative mechanisms to allow affected persons to seek and obtain correction of disseminated information that does not comply with OMB, HHS and AoA guidelines.

The AoA official with overall responsibility for receiving and resolving complaints regarding information that does not comply with agency's guidelines for data quality is the Director of the Center for Communications and Consumer Services (CCCS). The Director of CCCS will have overall responsibility for implementing AoA Information Quality Guidelines, and will work collaboratively with other units in AoA.

1. Responsibility of the Complainant

   To seek a correction of information disseminated by the agency, individuals should follow the procedures described below.

   1. A complaint or request for review and correction of information shall be in written hard copy or electronic form;
   2. it shall be sent to the agency by mail or electronic-mail (e-mail); and
   3. it shall state that an information quality  request for correction is being submitted.
       

   The complaint shall contain

   4. a detailed description of the specific material that needs to be corrected including where the material is located, i.e., the publication title, date, and publication number, if any, or the web site and web page address (url), or the speech title, presenter, date and place of delivery; and
   5. the specific reasons for believing the information does not comply with OMB, HHS or AoA guidelines and is in error and supporting documentation, if any ;
   6. the specific recommendations for correcting the information;
   7. a description of how the person submitting the complaint is affected by the information error; and
   8. the name, mailing address, telephone number, e-mail address, and organizational affiliation, if any, of the individual making the complaint.

   
   Complainants should be aware that they bear the "burden of proof" with respect to the necessity for correction as well as with respect to the type of correction they seek. All complaints shall be sent to:
    

   Director, Center for Communications and Consumer Services
   Administration on Aging
   330 Independence Avenue, SW
   Washington, DC 20201

   
   Alternatively, complaints submitted via electronic mail should be sent to: ASPE.InfoQuality@hhs.gov.

2. Responsibility of the Agency

   Based on a review of the information provided, the agency will determine whether a correction is warranted and if, so what action to take. The agency will respond to the requestor by letter or e-mail. The agency's response will explain the findings of the review and the actions that the agency will take, if any. The response will consider the nature and timeliness of the information involved and such factors as the significance of the correction on the use of the information and the magnitude of the correction. The response will describe how the complainant may request reconsideration. The agency will respond to all requests for correction within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.

3. Appeals

   If the individual submitting the complaint does not agree with the agency's decision (including the corrective action, if any), the complainant may send a written hard copy or electronic request for reconsideration within 30 days of receipt of the agency's decision. The appeal shall state the reasons why the agency response is insufficient or inadequate. Complainants shall attach a copy of their original request and the agency response to it, clearly mark the appeal with the words, "Information Quality Appeal," and send the appeal to the specific agency appeals address. The agency will respond to all requests for appeals within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date. The agency official who resolved the original complaint will not have responsibility for the appeal. Appeals will be decided by the Director, Center for Wellness and Aging and should be addressed to:

   
   Deputy Assistant Secretary for Policy and Programs
   Administration on Aging
   330 Independence Avenue, SW
   Washington, DC 20201

VII. Influential Scientific, Financial, and Statistical Information

The OMB Information Quality Guidelines require that "influential" scientific, financial, or statistical information in official Government documents must be based on studies that can be substantially reproduced if the original or supporting data were to be independently reanalyzed using the same methods. "Influential" when used in the phrase "influential scientific, financial, or statistical information" means that the AoA can reasonably determine that dissemination of the information will have or does have a clear and substantial impact on important public policies or important private sector decisions, or will have important consequences for specific health practices, technologies, substances, products, or firms.

AoA is committed to applying rigorous professional standards to ensure the accuracy and reliability of research results. For professional and technical documents, AoA regularly seeks input from qualified analysts and experts from both within and outside AoA to insure that these influential materials are appropriate, accurate, complete, and of high quality (including objective, useful, and valid) prior to dissemination.

AoA continues to encourage the sharing of data and methods where practicable. Since the influence and implications of AoA-disseminated information cannot always be fully anticipated, all AoA reports are expected to state clearly how analytic results are generated -- the specific data used, various assumptions, specific analytic methods, statistical procedures, sources of error -- making the analysis sufficiently transparent so as to be capable of being reproduced. AoA advocates the archiving of data where feasible to facilitate the reproducibility of influential information.

VIII. References

HHS Standards of Conduct Regulations (45 CFR 73.735-705) (updated October 1, 2000).

The HHS Printing Handbook (September 1998).

AoA Administrative Procedures Manual (2001).

AOA Grants Administration Manual.

Office of Management and Budget (OMB) "Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies" -- Final Guidelines (January 3, 2002).

OMB, Circular No. A-130, Revised (Transmittal Memorandum No. 4). Management of Federal Information Resources (November 30, 2000).

Last revised: December 13, 2006

I. Agency Mission

The mission of the Agency for Healthcare Research and Quality (AHRQ) is to develop scientific evidence that enables health care decision makers — patients and clinicians, those who manage and purchase health care services, and policy makers  — to make more informed health care choices. AHRQ accomplishes its mission by conducting, supporting, and disseminating scientific research designed to improve the outcomes, quality, and safety of health care, reduce its cost, broaden access to effective services, and improve the efficiency and effectiveness of the ways health care services are organized, delivered, and financed.

The Agency was created December 1989 as the Agency for Health Care Policy and Research (AHCPR) and reauthorized on December 6, 1999, as the Agency for Healthcare Research and Quality. AHRQ is an agency of the Public Health Service (PHS) within the U.S. Department of Health and Human Services (HHS).

At AHRQ, the Director will have overall responsibility for implementing the AHRQ Information Quality Guidelines.

II. Scope and Applicability of Guidelines for Agency

AHRQ will ensure that disseminated information meets the standards of quality set forth in the OMB, HHS and AHRQ guidelines. It is AHRQ's goal to ensure and maximize the quality, objectivity, utility, and integrity of information that it disseminates to the public. AHRQ strives to provide information that is accurate, reliable, clear, complete, unbiased, and useful. It is committed to integrating the principle of information quality into every phase of information development, including creation, collection, maintenance, and dissemination.

The pre-dissemination review described in the guidelines only applies to information disseminated on or after October 1, 2002. The administrative mechanism for correction applies to information that the agency disseminates on or after October 1, 2002, regardless of when the agency first disseminated the information.  The guidelines apply to information in all media (e.g., print, electronic) and apply primarily to the dissemination of substantive information (e.g., reports, studies, and summaries relating to the findings and related products of AHRQ research) rather than on information pertaining to basic Agency operations not intended for public dissemination.

Many AHRQ activities result in publicly disseminated information that is subject to the OMB Guidelines including:
 

• Statistical information and databases
• Results of research projects
• Program evaluations
• Software tools
• Consumer and professional health information

The following types of information produced by the Agency are not covered: scientific information produced by extramural researchers who conduct and disseminate their own work in the same manner as their academic colleagues and whose views do not represent the views of the Agency; information in which the results are identified as not being Agency views or positions; archival information; information intended soley for intra-or-interagency use or sharing of government information; press releases that support the announcement or give public notice of information that AHRQ has disseminated elsewhere; correspondence with individuals; responses to requests for Agency records under the Freedom of Information Act (FOIA); public filings; subpoenas; and the findings and determinations that AHRQ makes in the course of adjudications  involving specific parties.

III. Types of Information Disseminated by the Agency to the Public

Data products  — Includes print and electronic materials that describe or present aggregate statistical information such as data from the Medical Expenditure Panel Survey (MEPS), a survey of households regarding how they use and pay for health care. MEPS Public Use Data Files (on the Web site and CD-ROM) allow access to aggregate data. Another example is the Healthcare Cost and Utilization Project (HCUP), a family of health care databases and related tools with data provided by state data organizations, state hospital associations, and other sources. Web-based interactive analytic tools (i.e., MEPSnet, HCUPnet) are also available.

Results of research projects  — Research reports and other information products such as fact sheets, research syntheses, and AHRQ's monthly newsletter Research Activities are produced in both print and electronic formats and describe the findings and methods of AHRQ-funded research projects.

Program evaluation reports  — Reports summarizing the results of program evaluations are included in the departmental catalog of program evaluations maintained by the Office of the Assistant Secretary for Planning and Evaluation. Many of the Agency's program evaluations are driven by the goals established under the Government Performance and Results Act (GPRA).

Software tools  — Tools developed to assist health care policy makers, health system leaders, purchasers, and others in carrying out their tasks and making health care choices. An example is the National Guideline Clearinghouse (NGC), which provides standardized information on evidence-based clinical practice guidelines.

Consumer and professional health information  — AHRQ produces a wide range of print and electronic consumer and professional information. Examples include consumer publications on: various health conditions/diseases, on how to select a health plan, personal health guides, and how to improve the quality of health care you receive as a patient. Examples of publications for professionals include The Physicians Handbook of Preventive Services and evidence reports on a wide array of clinical topics that can be used by public and private entities to develop and implement their own practice guidelines, performance measures, review criteria, and other clinical quality improvement tools.

IV. Types of Dissemination Methods

AHRQ principally uses print and electronic methods of dissemination.

Print methods include:
 

• Reports, fact sheets, and research syntheses communicating the results of surveys and other research activities
• Publication of Research Activities, AHRQ's monthly newsletter AHRQ

Electronic methods include:
 

• The AHRQ Web site as well as AHRQ program specific sites
• AHRQ's Electronic Newsletter

Information Clearinghouse: AHRQ also has an Information Clearinghouse used to disseminate print and other media.

Secondary methods of dissemination include, but are not limited to: workshops, audiovisual techniques, oral presentations, and teleconferences. These dissemination mechanisms largely draw on information developed and reviewed first through one of the primary print or electronic modes of information dissemination.

V. Agency Quality Assurance Policies, Standards and Processes for Ensuring the Quality of Information Disseminated to the Public

The quality assurance process begins at the inception of the information development process. AHRQ reviews the quality (including the objectivity, utility, and integrity) of information before it is disseminated and treats information quality as integral to every step of the development of information, including creation, collection, maintenance, and dissemination. AHRQ will demonstrate in its Paperwork Reduction Act (PRA) clearance packages that each draft information collection will result in information that will be collected, maintained, and used in a way that is consistent with OMB, HHS and AHRQ information quality guidelines.

Work performed by AHRQ is conducted in compliance with generally accepted professional standards appropriate to the field(s) of study being undertaken, e.g., clinical, statistics, economics, or qualitative health services research. Moreover, work produced by the Agency undergoes many levels of review before being finalized and disseminated to ensure quality, i.e., utility, objectivity (in substance and presentation), and integrity. Reviews are both hierarchical, i.e., performed by higher level supervisors and managers, and by peer experts both inside and outside of the Agency. Generally, it is the responsibility of AHRQ's Center Directors to ensure the utility, objectivity, and integrity of work produced within their research Centers.

The discussion that follows outlines the general quality standards and quality assurance procedures that AHRQ has in place. More detailed information is available on AHRQ's Web site and that information will be expanded to reflect new products (e.g. new databases) or revised over time to reflect advances in generally accepted professional standards.

Statistical information and database development

Quality standards: Statistical activities are carried out in accordance with generally accepted professional standards and modern statistical theory and practice, including scientific sampling, statistical inference and analytical techniques and practices. All statistical programs employ or have access to experts in statistics and research design.

Quality assurance procedures: Each of the Agency's data products undergoes a series of standardized quality assurance procedures to assure the validity, reliability, and consistency of the data. These procedures are posted on the AHRQ Web site. For example, the Medical Expenditure Panel Survey (MEPS), which involves data collection, uses quality assurance procedures such as the validation of interviewers' work, performance of quality control checks on the source variables across all analytic groups, and benchmarking the data (e.g. comparisons with prior years and comparable data sources such as the Current Population Survey of the U.S. Census Bureau and the National Health Interview Survey). The Healthcare Cost and Utilization Project (HCUP), which aggregates and standardizes existing data from states, has quality control procedures in place to verify that the databases contain information that is valid, internally consistent, and consistent with established norms. Procedures include automated edit checks, careful review of all summary statistics, and benchmarking data with other sources (e.g. comparison with the National Hospital Discharge Survey).

Before disseminating information in any format, data are reviewed internally to ensure that they are scientifically sound and meet or exceed standards for data quality, statistical integrity, and confidentiality and privacy protections. Similarly, description and documentation of databases, tools, products, and on-line query systems are carefully reviewed, edited, and re-reviewed before release to the public.

Results of research projects

Quality standards: AHRQ staff adhere to the highest professional standards in their appropriate/respective fields, e.g., economics, social science and statistics. They seek to create transparency in their activities to enable others to fully understand their research processes, methodologies, and assumptions so that the work can be evaluated and replicated. Staff must ensure that strong measures are in place to guarantee the integrity of research information and to protect it from unauthorized access or revision.

Quality assurance procedures: All intramural research projects  — whether they involve qualitative or quantitative analysis  — and all projects on which AHRQ staff collaborate with authors outside the Agency are subject to AHRQ's intramural research planning process. This is intended to ensure conformance with strategic planning, accountability, peer review, scientific objectivity, confidentiality, and privacy standards. Research reports that are the products of intramural research undergo many levels of review prior to release, including hierarchical and peer reviews (sometimes including external reviews by other agencies and outside experts). Manuscripts submitted to scientific journals are subject to additional peer-review by the journal, prior to acceptance and publication.

In keeping with widely-accepted scientific research practices, research reports published by AHRQ describe the methods, data sources, analytical techniques, assumptions, and limitations of the research so the study can be substantially assessed and replicated.

Program Evaluations

Quality standards: Program evaluations are developed and assessed in accordance with the highest professional standards of evaluation practice.

Quality assurance procedures: Evaluation proposals are reviewed by Agency management for suitability, utility, and methodology, in accordance with sound evaluation design and standards of evaluation practice. Many evaluation projects have technical advisory committees that oversee the design and conduct of the evaluation in accordance with standard evaluation theory and practice; they often provide an expert review of the final report. Results of evaluation activities are released to the public only after Agency management has completed a review of the quality, accuracy and completeness of the report.

Software tools

Quality standards: Software tools are developed by teams of subject area experts in compliance with generally acceptable professional standards appropriate to their respective fields. Many levels of review and testing are built into the development process to ensure that the final product is useful, objective, and of high quality.

Quality assurance procedures: Reviews of tools in development are stringent. To ensure that they are needed and useful, the Agency generally relies upon peer experts and user groups both inside and outside of the Department. The Agency conducts extensive testing prior to release. Feedback is obtained from users of the tools or software to identify enhancements that will make the tools or software more fully meet user needs. As an example, the NGC is currently conducting its third annual on-line user survey to assess experiences with NGC and gather their recommendations for how to improve the system.

Consumer and professional health information

Quality standards: Consumer and professional information developed by AHRQ is conducted in compliance with generally accepted professional standards appropriate to the field of study being undertaken, e.g., clinical, economics, statistics, and social science research, as well as sound health care communication principles.

Quality assurance procedures: AHRQ works closely with stakeholder groups to determine the need for consumer and professional publications. Publications are based on information emanating from research that has been reviewed for scientific and medical accuracy and completeness by experts within and outside of the Agency before it is disseminated. Consumer publications are tested with targeted audiences to ensure relevance, clarity, and comprehension. In addition, AHRQ's publications containing health care recommendations indicate the level of substantiating evidence.

VI. Agency Administrative Complaint Procedures

AHRQ has developed administrative mechanisms to allow affected persons to seek and obtain correction of disseminated information that does not comply with OMB, HHS and AHRQ guidelines. Persons seeking correction of AHRQ-produced information should follow the procedures below.

1. Responsibility of the Complainant

   To seek a correction of information disseminated by the Agency, complainants should follow the procedures described below. Complaints or requests for review and correction of information shall be in written hard copy or electronic form. They shall be sent to the Agency by regular or electronic mail. Letters (written hard copy) should be sent to: 

   Office of the Director
   Eisenberg Building, Suite 3000
   540 Gaither Road
   Rockville, MD 20850 

   Electronic mail should be sent to Roger.Klein@ahrq.hhs.gov and a copy sent to ASPE.InfoQuality@hhs.gov. 

   Complaints should state "Information Quality Concern" in the subject line. The body of the complaint shall state that a request for correction of information is being submitted. Complaints shall contain a detailed description of the specific material that needs to be corrected including a detailed citation, e.g., the publication title, date, and publication number, if any, or the Web site and web page address (url). Complaints should describe the specific reasons for believing the information does not comply with OMB, HHS, or AHRQ guidelines and is in error and supporting documentation, if any. In addition, they should include specific recommendations for correcting the information; a description of how the person submitting the complaint is affected by the information error; and the name, mailing address, telephone number, e-mail address, and organizational affiliation, if any, of the individual or organization making the complaint. Requesters should be aware that they bear the "burden of proof" with respect to the necessity for correction as well as with respect to the type of correction they seek. In determining whether and how to address a request, the Agency will consider such factors as the Agency's budget, resources, and priorities, as well as the complexity of the correction task itself. Also, it needs to be understood that it may not be in the public interest for the Agency to devote significant resources to correcting information where the expenditure of such resources is not, in the Agency's view, cost effective in light of the significance of the asserted error, the benefits that are likely to be derived from such a correction, the costs of the correction, and the Agency's more pressing priorities and obligations.

2. Responsibility of the Agency

   Based on a review of the information provided, the Agency will determine whether a correction is warranted and if, so what action to take. The Agency will respond to the requestor by letter or e-mail. The Agency's response will explain the findings of the review and the actions that the Agency will take, if any. The response will consider the nature and timeliness of the information involved and such factors as the significance of the correction on the use of the information and the magnitude of the correction. The response will describe how the complainant may request reconsideration. The Agency will respond to all requests for correction within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the Agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.

3. Appeals

   If the individual submitting the complaint does not agree with the Agency's decision (including the corrective action, if any), the complainant may send a written hard copy or electronic request for reconsideration within 30 days of receipt of the Agency's decision. The appeal shall state the reasons why the Agency response is insufficient or inadequate. Complainants shall attach a copy of their original request and the Agency response to it, clearly mark the appeal with the words, "Information Quality Appeal," and send the appeal to the addresses for hard copy or electronic submissions identified above. The Agency will respond to all requests for appeals within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the Agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date. The Agency official who resolved the original complaint will not have responsibility for the appeal.

VII. Influential Scientific, Financial and Statistical Information

From time to time, AHRQ disseminates influential information, including statistical information, as defined in the OMB Guidelines. By "influential," OMB is referring to information "that the agency can reasonably determine...will have or does have a clear and substantial impact on important public policies or important private sector decisions." In accordance with standard Agency quality procedures, care is taken to ensure that information disseminated is substantially reproducible and replicable. This goal is accomplished by using reliable data sources and sound analytical techniques, and by employing a high degree of transparency about the data, methods, measures, assumptions and limitations used to develop the information in order to facilitate reproducibility by qualified third parties.

Because of ethical and feasibility constraints as well as confidentiality and legal obligations to third parties supplying information, there may be instances where not all of the original or supporting data may be available to the public. In such instances, AHRQ will promote understanding and transparency by clearly identifying, to the extent practicable, the resulting research methodologies and assumptions in reference documents.

I. Agency Mission

The Centers for Disease Control and Prevention (CDC) and the Agency for Toxic Substances and Disease Registry (ATSDR) are two of the operating components of the HHS. Although CDC and ATSDR are separate agencies, both strive to protect and improve the health of the American public. The Director of CDC also serves as the Administrator of ATSDR.

CDC is the nation’s health protection agency with the mission to protect America from health, safety and security threats, both foreign and in the United States. CDC is at the forefront of public health efforts to prevent and control infectious and chronic diseases, injuries, workplace hazards, disabilities, and environmental and occupational health threats. CDC accomplishes its mission by working with partners throughout the nation and world to monitor health, detect and investigate health problems, conduct research to enhance prevention, develop and advance sound public health policies, implement prevention strategies and programs, promote healthy behaviors, foster safe and healthful environments, and provide leadership and training.

CDC’s Centers, Institutes, and Offices (CIOs) include:

• Global Health Center (GHC)
• National Center on Birth Defects and Developmental Disabilities (NCBDDD)
• National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP)
• National Center for Emerging and Zoonotic Infectious Diseases (NCEZID)
• National Center for Environmental Health (NCEH)/ Agency for Toxic Substances and Disease Registry (ATSDR)
• National Center for Health Statistics (NCHS)
• National Center for HIV, Viral Hepatitis, STD, and TB Prevention (NCHHSTP)
• National Center for Immunization and Respiratory Diseases (NCIRD)
• National Center for Injury Prevention and Control (NCIPC)
• National Center for State, Tribal, Local, and Territorial Public Health Infrastructure and Workforce (NCSTLTPHIW)
• National Institute for Occupational Safety and Health (NIOSH)
• Center for Forecasting and Outbreak Analytics (CFA)
• CDC Washington Office (CDC/W)
• Office of Communications (OC)

ATSDR was established in 1980 by the Comprehensive Environmental Response, Compensation, and Liability Act, also known as Superfund. ATSDR works to prevent exposures to hazardous wastes and to environmental releases of hazardous substances. Working with States and other Federal agencies, ATSDR seeks to prevent exposure and adverse health effects associated with exposure to hazardous substances from waste sites. The agency conducts public health assessments, health studies, surveillance activities and health education training in communities around waste sites or exposed to environmental releases. ATSDR also develops toxicological profiles of hazardous chemicals found at these sites.

Unless otherwise specified, all subsequent references to CDC also include ATSDR. All practices and procedures described in this document apply to both agencies.

II. Scope and Applicability of Guidelines for CDC 

CDC will ensure that disseminated information meets the standards of quality set forth in the OMB, HHS and CDC guidelines. It is CDC's policy to ensure and maximize the quality, objectivity, utility, and integrity of information that it disseminates to the public. We strive to provide information that is accurate, reliable, clear, complete, unbiased, and useful. We are committed to integrating the principle of information quality into every phase of information development, including creation, collection, maintenance, and dissemination.

CDC guidelines do not apply to the National Center for Health Statistics (NCHS). While NCHS is a component of CDC, NCHS is the nation's principal health statistics agency and as such has separate guidelines.

The pre-dissemination review described in the guidelines only applies to information disseminated on or after October 1, 2002. The administrative mechanism for correction applies to information that the agency disseminates on or after October 1, 2002, regardless of when the agency first disseminated the information.

The guidelines apply to information in all media — print, electronic, audiovisual, and oral. They apply to substantive information, such as studies and reports, rather than to information pertaining to basic agency operations. Information that is disseminated at the request of CDC or with specific CDC approval through a contract, a grant, or a cooperative agreement is subject to these guidelines.

Examples are provided below of the types of information that the CDC considers within and outside the scope of the guidelines.

A. Covered Information

• Scientific research papers, books, journal articles, reports, and similar materials, unless they have disclaimers to distinguish the research from CDC views and positions;
• Other official reports, brochures, documents, newsletters, and audiovisual products;
• Oral information, including speeches, interviews, expert opinions only if representing CDC's views, official positions, or policies;
• Statistical information - statistical analyses, aggregated information by programs.

B. Information Not Covered

• Documents not authored by CDC (either directly or by contract) and not representing official views, including research and science supported by CDC funding;
• Opinions where the presentation makes it clear that what is being offered is personal opinion rather than fact or CDC's views;
• Archival information disseminated by CDC (for example, Internet distribution of published articles);
• Information dissemination limited to government employees or agency contractors or grantees;
• Information intended solely for intra- or inter-agency use or sharing of government information, such as evaluation of a specific public health program to assess the success in achieving its objectives, technical assistance reports, training materials, manuals;
• Information intended to be limited to public filings, subpoenas, or adjudicative processes;
• Press releases that support the announcement or give public notice of information that CDC has disseminated elsewhere.
• Information pertaining to basic agency operations (e.g., information about agency authorities, activities, programs, along with contact information for the public, organizational charts) CDC or CIO Directors' Status Reports, solicitations [e.g., Notice of Funding Opportunity (NOFO)] and related receipt and review materials (e.g., summary statements)
• Information that is limited in dissemination to Government employees or agency contractors or grantees
• Responses to requests for agency records under the Freedom of Information Act (FOIA), the Privacy Act, the Federal Advisory Committee Act, or other similar laws
• Information relating solely to correspondence with individuals or persons

III. Types of Information Disseminated by CDC to the Public 

Annually, CDC produces hundreds of publications of various types and provides Web content for access by the public. All publications that carry the CDC logo are considered official publications or releases, and must follow CDC policy and procedures for preparation, review, approval, and distribution.

Examples of the types of information disseminated by CDC to the public are listed below. Some types fit into more than one category and are mentioned in each.

1. Scientific research studies.
   CDC encourages professional dissemination of scientific research by employees and those funded by CDC to conduct research. These research studies may be published by CDC, such as the Morbidity and Mortality Weekly Report (MMWR) or non-CDC publications including journals, books, chapters, editorials, reviews, proceedings or abstracts. These are usually authored by or co-authored by CDC staff scientists as part of their official duties or may be authored by CDC partners, CDC advisory committees, or working groups convened by CDC.
2. Statistical products
   CDC releases data sets and disseminates statistical reports produced by its data collection programs. These include vital statistics, population-based health surveys, and surveys of health care providers.
3. Programmatic and administrative information.
   CDC disseminates community health assessments and information in connection with and as a byproduct of the administration of programs, such as Program-in-Brief documents, At-A-Glance documents, and program brochures.
4. Authoritative health, medical and human services information aimed at consumers and health and human services professionals.
   CDC publishes the MMWR, which includes Recommendations and Reports. CDC generates Health Alerts, Public Health Advisories, and guidelines for dealing with specific public health threats. CDC also provides the website Travelers' Health, which publishes guidelines for international travelers including the "Yellow Book" and official expert opinions. CDC produces and broadcasts science educational materials and training modules, including Vital Signs, and webcasts.
5. Public heath surveillance and epidemiology information.
   CDC publishes the MMWR Summary of Notifiable Diseases and CDC Surveillance Summaries, and other surveillance summaries on a variety of infectious diseases and other non-infectious conditions. The latest surveillance and epidemiological data are available through a number of systems (https://www.cdc.gov/surveillancepractice/index.html). In addition, CDC publishes outbreak investigations or other items reported in the MMWR that are not authoritative or urgent. ATSDR disseminates information products including public health assessments, public health consultations, fact sheets, health study reports, toxicological profiles, and Case Studies in Environmental Medicine.

IV. Types of Dissemination Methods 

CDC disseminates information through a wide range of methods, often using more than one medium for the same information.

1. Print
   Publications in peer-reviewed literature, published reports, periodicals, brochures, books, and correspondence
2. Electronic
   CDC Website, social media, Listserv, e-mail, automated voice, and hotlines
3. Audiovisual
   Broadcast scripts, audio or videotapes, and videocasting
4. Oral
   Formal speeches, oral presentations, and interviews, or commentaries for publication or broadcast

V. Agency Quality Assurance Policies, Standards and Processes for Ensuring the Quality of Information Dissemination to the Public. 

1. Overview
    

   CDC's policies and procedures are designed to ensure and maximize the quality of its information products with regard to their utility, objectivity, and integrity. The agency's quality assurance process begins at the inception of the information development process. CDC has guidelines to address the general principles concerning the responsibilities of the CDC staff in the collection and recording of data, publication practices, authorship determination, peer review, confidentiality of information, collaborations, and human subjects research. Authorship issues and review and clearance procedures are set forth in the "Authorship of CDC Publications and the Clearance Procedures for Scientific and Technical Documents".

   CDC reviews the quality (including the objectivity, utility, and integrity) of information before it is disseminated and treats information quality as integral to every step of the development of information, including its creation, collection, maintenance and dissemination. Further, CDC is committed to demonstrating in its Paperwork Reduction Act clearance packages that each draft information collection will result in information that will be collected, maintained, and used in a way that is consistent with OMB, HHS and CDC information quality guidelines. The individual CIO Associate Directors for Science (ADS) or designee are responsible for assuring the quality of information disseminated by CDC and that the quality assurance methods and procedures described in Overview of Quality Assurance Policies and Practices in HHS are met. To meet the standards for external merit review of research and scientific studies and intramural research programs, CDC policy is to peer review extramural research studies and programs.

   The CIO ADS or designee are responsible for clearance of documents originating in that CIO before dissemination and for ensuring that the necessary clearances are obtained and that written material distributed is appropriate and consistent with HHS policy. While each CIO can determine preparation, review and approval procedures, all must meet standards provided by the Director, Office of Science, CDC, and those provided in the HHS Guidelines.

   • Utility CDC addresses utility, a measure of the usefulness of information products to its intended users, by staying informed of user needs through information product research and user needs assessment, user feedback, consultation with advisory committees, and conference participation.
   • Objectivity CDC provides assurance that information is accurate, reliable, and unbiased. Objectivity is achieved through existing review and clearance procedures and, in many cases, the peer review of disseminated information.
   • Integrity CDC assures the integrity of its data and information products through the enforcement of rigorous controls that protect against unauthorized access, revision, or corruption. Some of the controls used at CDC include access control, user authentication, encryption, access monitoring, provision of unalterable electronic content, and audit trails.

   
    

2. CDC Information Review and Approval Policies and Procedures by Type of Information
    

   1. Health and Public Health Information
      1. Scientific research studies
         CDC encourages professional dissemination of scientific research and other information by its employees. Publications or presentations by CDC employees are expected to meet high standards of quality, make a substantial contribution to the field, and contain sufficient information for the informed audience to assess its validity. Publication of scientific information by individual employees must undergo a formal review and clearance process by the CIO ADS or designee before dissemination. This review includes the evaluation of data collection measures for completeness, accuracy and timeliness, data management and analysis, clarity and accuracy of presentation, and validity of interpretation of findings.
         Oral presentations undergo appropriate supervisory review. Laboratory data are reviewed to assure that good laboratory data practice was followed for sampling, methodology, instrumentation, and analysis.
         Intramural research programs may be subject to review and monitoring by external, objective peer review through an advisory committee or board of scientific counselors. Scientific research studies submitted to journals are subject to peer review of methods and findings by the journal prior to publication. ATSDR has a mandated policy for external peer review of Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA) funded intramural and extramural research study protocols and findings prior to public dissemination.
      2. Authoritative health, medical and human services information aimed at consumers and health and human services professionals
         CDC disseminates authoritative health and medical information routinely as part of its mission. As an example, articles or reports for publication in the MMWR are subject to routine CDC review and approval procedures in the originating CIO. Because information disseminated in the MMWR often has impact on the practice of public health, the Director, Office of Science, CDC must also review and approve it. Health Alerts that are disseminated by CDC are also reviewed and approved at the Director, Office of Science, CDC level prior to release.
      3. Public heath surveillance and epidemiology information
         CDC often obtains surveillance information from third parties, such as States, grantees, or community-based organizations. Reliance on third parties places limits on CDC's quality assurance, although the accuracy, completeness and timeliness of the information are subject to sample audits, site visits, and an evaluation for completeness and consistency with trends and external controls. The MMWR Summary of Notifiable Diseases, for example, depends on data reported from States. CDC conducts audits and checks for consistency for trends before reporting these data. ATSDR produces toxicological profiles for hazardous substances found at National Priorities List sites as well as other documents that undergo public comment periods before being finalized and distributed. The toxicological profiles and other ATSDR documents are first produced as drafts and are then subject to public comments following announcement in the Federal Register and/or using other means. Only after considering the comments, the profiles and documents are finalized and then distributed to the public.
         ATSDR works with and respects Tribal rights, sovereignty, and culture. Data or information collected from American Indian/Alaska Native communities typically requires approval from the Tribal government and direct involvement in the research or study from concept to completion. The Tribe reserves the right to review and critique the design and findings. Issues of release and ownership of data, information or other products must be agreed to by the Tribal government. Close collaboration and involvement of the Tribe is essential to ensure quality, utility, objectivity, and integrity of information prior to being disseminated.
         
          

   2. Statistical products

      CDC routinely employs a number of widely accepted methods and procedures for ensuring quality, including independent assessments of statistical methodologies, peer reviews, and observance of professional standards. To insure the utility of CDC statistical and analytic information products, CDC conducts independent research and consults experts in areas such as data collection, data analysis and a variety of substantive topics and areas. Additionally, CDC maintains ongoing contact with users, and participates in conferences, and workshops in order to objectively assess and identify the current and future data needs of CDC's constituents. Further, CDC employs a wide variety of dissemination mechanisms to make its statistical and analytic information products widely available and broadly accessible.

      To assure that statistical and analytic information products are accurate, reliable, and unbiased, CDC obtains these data through generally accepted statistical theory and practice. Dissemination of data also follows generally recognized guidelines in terms of defining acceptable standards regarding minimum response rates, maximum standard errors, cell size suppression, quality of coding and other processing operations. CDC will continue to use existing data sources from both inside and outside the agency for statistical and research purposes, applying safeguards to protect privacy and confidentiality. CDC staff performing secondary analysis of data that include personally identifiable information are required to coordinate with the CDC Senior Agency Official for Privacy to meet all privacy requirements and manage privacy risks.

      CDC also maintains staff expertise in areas such as concept development, survey planning and design, data collection, data processing and editing, data analysis, evaluation procedures, and methods of data dissemination.

      When CDC makes information originally collected or developed by other Federal agencies available to the public in a cross-agency dissemination, each agency will be responsible for the quality of the information they contribute.

      CDC will provide the public with sufficient documentation about each dataset released to allow data users to determine the fitness and quality of the data for their intended use. As appropriate, CIOs may develop standardized templates or procedures to ensure potential users have the relevant information about data quality and fitness for statistical purposes.

      When designing or improving data collection systems, CDC staff may solicit comment from NCHS, or research and evaluation components of the agency, about potential downstream uses. Such uses will also be described in the Information Collection Request submitted to 0MB for review under the P RA.

      All CDC statistical and analytic information products undergo a formal clearance process before dissemination. Publications and reports, whether in electronic or paper form, are reviewed by a CIO ADS or designee. These reviews cover the clarity of descriptive text, the appropriateness of the methodology, the soundness of the analysis, the adherence to confidentiality and disclosure avoidance restrictions, the readability of tabular and graphic presentations of data. Oral presentations undergo appropriate supervisory review. The CIO ADS or designee may also review for programmatic and policy implications on behalf of and in consultation with other division or senior staff.

      CDC statistical and analytic information products are derived using generally acceptable statistical practices and methodologies which are clearly documented and available to the public. These procedures enable statisticians and analysts outside of CDC to replicate CDC's statistical methods and obtain results consistent with those obtained by CDC. When conducting peer review, agencies should ensure reviewers are asked to evaluate the objectivity of the underlying data and the sensitivity of the agency's conclusions to analytic assumptions.

   VI. Agency Administrative Complaint Procedures

   CDC has developed administrative mechanisms to allow affected persons to seek and obtain correction of disseminated information that does not comply with OMB, HHS and CDC guidelines.

   CDC maintains a Website to advise information consumers of the agency's information quality guidelines, the process to submit a complaint, information needed by the complainant, and a description of the complaint adjudication process. CDC centralizes the receipt, logging, and tracking of complaints received under this provision in the Office of Science Quality and Library Services within the Office of Science. Complaints are forwarded to the office that has subject matter responsibility for the information product in question.

   1. Responsibility of the Complainant
       

      To seek a correction of information disseminated by the agency, individuals must follow the procedures described below:
       

      Complaints or requests for review and correction of information must be in written (hard copy or electronic) form; requests shall be sent to CDC by mail to:
          CDC/ATSDR, (Attn.: Office of Science Quality)
          1600 Clifton Road, N.E., 
          Mailstop MS-H19-1, 
          Atlanta, GA 30333
      or by e-mail at: InfoQuality@cdc.gov; and requests should state that an information quality request for correction is being submitted.

      
      The complaint must contain:

      1. a detailed description of the specific information that needs to be corrected including where the information is located, i.e. the publication title, date, and publication number, if any, or the Website and Web page address (url), or the speech title, presenter, date and place of delivery;
      2. the specific reasons for believing the information does not comply with OMB, HHS or CDC guidelines and is in error and supporting documentation, if any;
      3. the specific recommendations for correcting the information;
      4. a description of how the person submitting the complaint is affected by the information error; and
      5. the name, mailing address, telephone number, e-mail address, and organizational affiliation, if any, of the individual making the complaint.

      Complainants should be aware that they bear the 'burden of proof' with respect to the necessity for correction as well as with respect to the type of correction they seek.

   2. CDC/ATSDR Responsibility
       

      In alignment with the OMB Memorandum 19-15, CDC will respond to all requests for correction within 30 calendar days of receipt. If the request requires more than 30 calendar days to resolve, the requestor will be informed that more time is required, notified of the reason why, and provided an estimated decision date. CDC will take no longer than 120 days to respond to the request for correction without the concurrence of the party requesting a correction. Based on a review of the information provided, the agency will determine whether a correction is warranted and, if so, what action to take. CDC will respond to the requestor by letter or e-mail, explaining the findings of the review and the actions that the agency will take, if any. The response will consider the nature and timeliness of the information involved and such factors as the significance of the correction on the use of the information and the magnitude of the correction. The response will describe how the complainant may request reconsideration of the CDC decision.

   3. Appeals
       

      If the individual submitting the complaint does not agree with CDC's decision (including the corrective action, if any), the complainant may send a written hard copy or electronic request for reconsideration within 30 days of receipt of the agency's decision. The appeal must state the reasons why the agency response is insufficient or inadequate. Complainants must attach a copy of their original request and the agency's response to it. Clearly mark the appeal with the words, "Information Quality Appeal," and send the appeal by mail to CDC/ATSDR, CDC/ATSDR, (Attn.: Office of Science Quality and Library Services); 1600 Clifton Road, N.E., Mailstop MS-H19-1, Atlanta, GA 30333 or by e-mail to InfoQuality@cdc.gov.

      The agency will respond to all requests for appeals within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.

      The agency official who resolved the original complaint will not have responsibility for the appeal. CDC’s Office of Science Quality will direct all appeals to an appropriate CDC official based on the nature of the information product and complaint.

   VII. Influential Scientific, Financial and Statistical Information 

   “Influential”, when used in the phrase “influential scientific, financial, or statistical information” or “highly influential scientific assessment” means CDC has a high degree of certainty that dissemination of the information will have or does have a clear and substantial impact on important public policies or private sector decisions or will have important consequences for specific health practices, technologies, substances, produces, or firms. This designation applies only when there could be a substantial economic impact and the “influential” information can reasonably be viewed as one of the chief factors that substantially accounts for the economic impact. The impact must also affect a broad range of parties or sites. According to OMB guidance, “in the context of a policy decision, a specific piece or body of information is "influential" when it is a principal basis for a decision by a federal decision maker, that is, if the same decision would be difficult to reach in that information's absence or if the decision would lose its fundamental scientific, financial, or statistical underpinnings absent the information. Even if a decision is very important, a particular piece of information supporting it may or may not be ’influential,’ depending on whether the decision could be reached in the information's absence.”

   The designation of influential is applied only to major agency information products that are primarily scientific and technical in nature. Examples of influential information disseminated by CDC could include MMWR Recommendations and Reports, toxicological profiles, and ATSDR public health assessments. The designation is not applied to non-major information products or work products that lack substantial scientific or technical content or to derivative and secondary work products such as fact sheets, conference proceedings, bibliographies, brochures, speeches, and Congressional testimony given by HHS officials. CDC is committed to applying rigorous scientific standards to ensure the quality (utility, integrity, and objectivity), accuracy, reliability, and reproducibility of influential information.

   The term “scientific assessment” means an evaluation of a body of scientific or technical knowledge, which typically synthesizes multiple factual inputs, data, models, assumptions, or applies best professional judgment to bridge uncertainties in the available information. These assessments include, but are not limited to, state-of-science reports; technology assessments; weight-of-evidence analyses; meta-analyses; health, safety, or ecological risk assessments; toxicological characterizations of substances; integrated assessment models; hazard determinations; or exposure assessments. Such assessments often draw upon knowledge from multiple disciplines. Typically, the data and models used in scientific assessments have already been subject to some form of peer review.

   Consistent with the Office of Science and Technology Policy's 2010 Memorandum for the Heads of Executive Departments and Agencies: Scientific Integrity, CDC ensures that influential information is communicated transparently by including a clear explication of underlying assumptions, accurate contextualization of uncertainties, and a description of the probabilities associated with both optimistic and pessimistic projections, including best-case and worst-case scenarios.

   CDC prioritizes increased access to the data and analytic frameworks (e.g., models) used to generate influential information. All data disclosures must be consistent with statutory, regulatory, and policy requirements for protections of privacy and confidentiality, proprietary data, and confidential business information.

   CDC is committed to exploring methods that provide wider access to datasets while reducing the risk of disclosure of personally, identifiable information. In particular, tiered approaches are encouraged to make data widely available while protecting privacy. Implementation of such approaches must be consistent with principles for ethical governance, which include employing sound data security practices, protecting individual privacy, maintaining promised confidentiality, and ensuring appropriate access and use.

   When performing an analysis using a specialized set of computer code, the computer code used to process will be made available to the public for further analysis, if consistent with applicable law and policy. When using non-government sources to create influential information sufficient information on the characteristics of the data and analysis, including its scope (e.g., temporal or demographic), generation protocols, and any other information necessary to allow the public to reproduce CDC’s conclusions will be provided.

   Risk Assessment

   Some of the influential information that we disseminate is based on an analysis of the risks to the public of certain actions or exposures to hazardous substances. For purposes of this guidance, we are defining risk as the likelihood that injury or damage is or can be caused by a substance, technology, or activity. We use risk analysis (the integration of risk assessment with risk management and risk communication) as a tool to enhance the scientific basis for all of our regulatory decisions.

   The OMB Guidelines provide special considerations that must be taken into account in certain risk assessments, those that provide the basis for the dissemination of influential information. The guidelines state that "With regard to analysis of risks to human health, safety, and the environment maintained or disseminated by the agencies, agencies shall either adopt or adapt the quality principles applied by Congress to risk information used and disseminated pursuant to the Safe Drinking Water Act Amendments of 1996 (SDWA) (42 U.S.C. 300g-1(b)(3)(A) and (B))."

   The SDWA risk assessment principles are as follows:

   1. To the degree that the agency action is based on science, the agency shall use
      1. the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices
      2. data collected by accepted methods (if reliability of the method and the nature of the decision justify use of the data)
   2. In the dissemination of public information about risks, the agency shall ensure that the presentation of information about risk effects is comprehensive, informative, and understandable.
   3. In a document made available to the public in support of a regulation, the agency shall specify, to the extent practicable
      1. Each population addressed by any estimate of applicable risk effects
      2. The expected risk or central estimate of risk for the specific populations affected
      3. Each appropriate upper-bound or lower-bound estimate of risk
      4. Each significant uncertainty identified in the process of the assessment of risk effects and the studies that would assist in resolving the uncertainty and
      5. Peer-reviewed studies known to the agency that support, are directly relevant to, or fail to support any estimate of risk effects and the methodology used to reconcile the inconsistencies in the scientific data

   Many of our actions are based on scientific experts' judgments using available data, are essentially qualitative and do not lend themselves to the types of quantitative risk assessments contemplated by the SDWA principles. As a result, we have adapted the general principles for risk assessments from the SDWA to fit these situations.

   1. The agency will use
      1. the best available science and supporting studies conducted in accordance with sound and objective scientific practices, including peer-reviewed science and supporting studies when available
      2. data collected by accepted methods (if reliability of the method and the nature of the decision justify use of the data)
   2. In the dissemination of public information about risks, the agency will ensure that the presentation of information about risk effects is comprehensive, informative, and understandable.

   In situations where a quantitative risk assessment is conducted, we generally follow basic risk assessment principles in the NAS paradigm of 1983. Our needs for quantitative risk assessments range over a wide variety of hazards including physical hazards encountered during exposure to toxic substances and antimicrobial resistance to antibiotic therapy. Thus, we also ascribe to the statement from NAS when it revisited the risk assessment process in 1994 (Science and Judgment in Risk Assessment, NAS 1994): "Risk assessment is not a single process, but a systematic approach to organizing and analyzing scientific knowledge and information." In each of the areas we regulate, we apply risk assessment practices to the specific task that are widely accepted among relevant domestic and international public health agencies.

   For quantitative risk assessments in support of the dissemination of influential information, CDC intends to apply the following principles:

   1. The agency will use
      1. the best available science and supporting studies conducted in accordance with sound and objective scientific practices, including peer-reviewed science and supporting studies when available.
      2. data collected by accepted methods (if reliability of the method and the nature of the decision justifies use of the data).
   2. In the dissemination of public information about health risks, the agency shall ensure that the presentation of information is comprehensive, informative, and understandable, within the context of its intended purpose.
   3. In a document made available to the public, the agency shall specify, to the extent practicable-
      1. Each population addressed by any estimate of applicable effects;
      2. The expected or central estimate of risk for the specific populations affected;
      3. Each appropriate upper-bound and/or lower-bound risk estimates;
      4. Data gaps and other significant uncertainties identified in the process of the risk assessment and the studies that would assist in reducing the uncertainties; and
      5. Additional studies not used in the risk assessment that support or fail to support the findings of the assessment and the rationale of why they were not used.

   
   VIII. Special Considerations for Agency Dissemination 

   Special consideration also applies to information products that are urgent in nature and because of the potential risk to human health and safety, certain information products may be disseminated in an expedited manner without having fully complied with all normal quality guidelines; however, basic quality principles and processes will still apply and be followed.

   National Center for Health Statistics
   
   I. Agency Mission 

   The National Center for Health Statistics (NCHS), a component of the Centers for Disease Control and Prevention (CDC), is one of the federal statistical agencies belonging to the Interagency Council on Statistical Policy (ICSP). The ICSP, which is led by the Office of Management and Budget (OMB), is comprised of the heads of the nation's ten principal statistical agencies plus the heads of the statistical units of four non-statistical agencies. The ICSP coordinates statistical work across organizations, enabling the exchange of information about organization programs and activities, and provides advice and counsel to OMB on statistical activities. The statistical activities of these agencies are predominantly the collection, compilation, processing or analysis of information for statistical purposes. Within this framework, NCHS functions as the federal agency responsible for the collection and dissemination of the nation's vital and health statistics. Its mission is to provide statistical information that will guide actions and policies to improve the health of the American people.

   To carry out its mission, NCHS conducts a wide range of annual, periodic, and longitudinal sample surveys and administers the national vital statistics systems. As the nation's principal health statistics agency, NCHS leads the way with accurate, relevant, and timely data. To assure the accuracy, relevance, and timeliness of its statistical products, NCHS assumes responsibility for determining sources of data, measurement methods, methods of data collection and processing while minimizing respondent burden; employing appropriate methods of analysis, and ensuring the public availability of the data and documentation of the methods used to obtain the data. Within the constraints of resource availability, NCHS continually works to improve its data systems to provide information necessary for the formulation of sound public policy. As appropriate, NCHS seeks advice on its statistical program as a whole, including the setting of statistical priorities and on the statistical methodologies it uses. NCHS strives to meet the needs for access to its data while maintaining appropriate safeguards for the confidentiality of individual responses.

   II. Scope and Applicability of Guidelines 

   NCHS will ensure that disseminated information meets the standards of quality set forth in the OMB, HHS and NCHS guidelines. It is NCHS's policy to ensure and maximize the quality, objectivity, utility, and integrity of information that it disseminates to the public. We strive to provide information that is accurate, reliable, clear, complete, unbiased, and useful. We are committed to integrating the principle of information quality into every phase of information development, including creation, collection, maintenance, and dissemination.

   The pre-dissemination review described in the guidelines only applies to information disseminated on or after October 1, 2002. The administrative mechanism for correction applies to information that the agency disseminates on or after October 1, 2002, regardless of when the agency first disseminated the information.

   The guidelines cover all NCHS statistical and analytic information products originated by the agency, except as specified below:

   • Documents not authored by NCHS (either directly or by contract) and not representing official views, including research supported by NCHS funding;
   • Archival information disseminated by NCHS;
   • Information dissemination limited to government employees or agency contractors or grantees;
   • Information intended solely for intra- or inter-agency use or sharing of government information, such as evaluation of a specific methodological approach to assess success in achieving its objectives;
   • Press releases that support the announcement or give public notice of information that NCHS has disseminated elsewhere.
   • Information for public filings, subpoenas, or adjudicative processes;
   • Opinions where the presentation makes it clear that what is being offered is personal opinion rather than fact or the agency's views.

   
   III. Types of Information Disseminated by NCHS to the Public 

   NCHS disseminates a variety of statistical and analytic information products. These include: statistical reports on topics such as births, deaths, life tables, health status, health services utilization and the provision of health care; statistical tabulations; publications in its "Healthy People 2010" series; "Health, US," the Secretary's annual report to the Congress; and public and restricted use data files. NCHS also disseminates the results of epidemiologic, demographic and methodological research.

   IV. Types of Dissemination Methods

   NCHS statistical and analytic information products are distributed in one or a combination of the following media:

   • Print — publications, reports, books, brochures, and pamphlets.
   • Electronic — the NCHS website, CD ROMS, listservs, e-mail, automated voice and fax systems, hotlines and clearinghouses.
   • Oral — speeches, presentations, commentaries for publication or broadcast, etc.
   • Audio-Visual — broadcast scripts, audio or videotapes, and videocasting.

   
   V. Agency Quality Assurance Policies, Standards and Processes for Ensuring the Quality of Information Dissemination to the Public 

   NCHS is recognized as one of the major statistical agencies in the federal government. In developing its statistical and analytic information products, NCHS recognizes that the quality assurance process begins at the inception of the information development process routinely employs a number of widely accepted methods and procedures for ensuring quality, including independent assessments of statistical methodologies and observance of professional standards. In order to ensure the utility of its statistical and analytic information products, NCHS conducts independent research and consults with experts in areas such as data collection, data analysis and a variety of substantive topics and issues. Additionally, NCHS maintains ongoing contact with users, and participates in conferences, workshops, etc., in order to objectively assess and identify the current and future data needs of NCHS constituents. NCHS reviews the quality (including the objectivity, utility, and integrity) of information before it's disseminated and treats information quality as integral to every step of the development of information, including its creation, collection, maintenance and dissemination. Further, NCHS employs a wide variety of dissemination mechanisms to make its statistical and analytic information products widely available and broadly accessible.

   In order to assure the objectivity of its statistical and analytic information products, i.e., that they are accurate, reliable, and unbiased, NCHS obtains these data through accepted statistical theory and practice. Dissemination of data also follows generally recognized guidelines in terms of defining acceptable standards regarding minimum response rates, maximum standard errors, cell size suppression, quality of coding and other processing operations. NCHS also maintains staff expertise in areas such as concept development, survey planning and design (including questionnaire development and testing), data collection, data processing and editing, data analysis, evaluation procedures, and methods of dissemination. Further, NCHS strives to demonstrate in its Paper Reduction Act (PRA) clearance packages that each draft information collection will result in information that will be collected, maintained, and used in a way that is consistent with OMB, HHS and NCHS information quality guidelines.

   NCHS assures the security of its statistical and analytic information products through the enforcement of rigorous controls that protect against unauthorized access to the data, revision or corruption of the data, or unauthorized use of the data. Some of the major controls used at NCHS include access control, user authentication, encryption, access monitoring, provision of unalterable electronic content, and audit trails.

   All NCHS statistical and analytic information products, undergo a formal clearance process before dissemination. Publications and reports, whether in electronic or paper form, are reviewed by a designated official within the author's office or division and by the Associate Director for Science (ADS). These reviews cover the clarity of descriptive text, the appropriateness of the methodology, the soundness of the analysis, the adherence to confidentiality and disclosure avoidance restrictions, the readability of tabular and graphic presentations of data, etc. Finally, all products undergo editorial review (e.g., formatting, proofreading, spell checks, proper punctuation, etc.,) In addition, all public use tapes are reviewed for accuracy and appropriate confidentiality protections. Oral presentations are subject to appropriate supervisory review.

   NCHS statistical and analytic information products are derived using generally acceptable statistical practices and methodologies which are well documented and available to the public. These procedures enable responsible statisticians and analysts outside of NCHS to replicate the NCHS statistical methods and obtain results consistent with those obtained by NCHS.

   VI. Agency Administrative Complaint Procedures 
   A. Responsibility of the Complainant

   NCHS has developed administrative mechanisms to allow affected persons to seek and obtain correction of disseminated information that does not comply with OMB, HHS and NCHS guidelines.

    

   To seek a correction of information disseminated by NCHS, individuals should follow the procedures described below.

   A complaint or request for review and correction of information shall be in written hard copy or electronic form;
   it shall be sent to NCHS by mail or electronic-mail (e-mail); and
   it shall state that an information quality request for correction is being submitted.

   The complaint shall contain:

   1. a detailed description of the specific material that needs to be corrected, including where the material is located, i.e., the publication title, date, and publication number, if any, or the website and web page address (url), or the speech title, presenter, date and place of delivery;
   2. the specific reason(s) for believing the information does not comply with OMB, HHS or NCHS guidelines and is in error and supporting documentation, if any;
   3. the specific recommendations for correcting the information;
   4. a description of how the complainant is affected by the information error; and
   5. the name, mailing address, telephone number, e-mail address, and organizational affiliation, if any, of the complainant.

   Complainants should be aware that they bear the "burden of proof" with respect to the necessity for correction as well as with respect to the type of correction they seek.

   The complaint or request for review and correction of information shall be addressed to: 
    

   Office of Science Quality and Library Services)
   1600 Clifton Road, N.E., 
   Mailstop MS-H19-1
   Atlanta, GA 30333

   
   or by e-mail to InfoQuality@cdc.gov.

    

   B. Responsibility of NCHS

   Based on a review of the information provided, NCHS will determine whether a correction is warranted and, if so, what action to take. NCHS will respond to the requester by letter or e-mail. The NCHS response will explain the findings of the review and the actions that it will take, if any. The response will consider the nature and timeliness of the information involved and such factors as the significance of the correction on the use of the information and the magnitude of the correction. The response will describe how the complainant may request reconsideration. NCHS will respond to all requests for correction within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, NCHS will inform the complainant that more time is required and indicate the reason(s) why. NCHS will also provide an estimated decision date.

   C. Appeals

   If the complainant does not agree with the NCHS decision (including the corrective action, if any), the complainant may send a written hard copy or electronic request for reconsideration within 30 days of receipt of the NCHS decision. The appeal shall state the reason(s) why the NCHS response is insufficient or inadequate. Complainants shall attach a copy of their original request and the NCHS response to it, clearly mark the appeal with the words, "Information Quality Appeal," and send the appeal to:

   Office of Science Quality and Library Services)
   1600 Clifton Road, N.E., 
   Mailstop MS-H19-1
   Atlanta, GA 30333

   
   or by e-mail to InfoQuality@cdc.gov.

    

   The NCHS official who reviewed the original complaint will not have responsibility for responding to the appeal. The agency will respond to all requests for appeals within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.

   VII. Influential Scientific, Financial and Statistical Information

   NCHS statistical and analytic data information products are used extensively to evaluate and profile the health of the American people. Often, findings from NCHS surveys serve as benchmarks for investigating the need to modify or implement policies that will affect initiatives addressing specific health concerns of the United States population or specific health concerns of certain population sub-groups. In recognition of the potential impact some of its statistical and analytic data products may have on health policy and initiatives, NCHS requires that these products meet the highest possible quality standards. In order to accomplish this goal, quality standards are embedded throughout all survey processes and procedures, beginning with the conceptualization and design of a survey, through to data dissemination. The appropriateness of many of these standards is measured by independent assessments (e.g., peer review) conducted by qualified persons. Further, NCHS strives to assure that its processes and procedures are appropriate, well documented, and capable of being replicated by qualified persons outside of NCHS in order to derive results consistent with those obtained by NCHS. Whenever confidentiality restrictions preclude the general availability of NCHS data sets, methodologies, etc., NCHS makes every effort to provide qualified researchers limited access to confidential files through its Research Data Center, which was established specifically for that purpose.

I. Agency Mission

The Centers for Medicare & Medicaid Services (CMS) administers the Medicare program, providing health care security and choice for aged and disabled people in this country. Jointly with the State governments, CMS administers the Medicaid program and the State Children's Health Insurance Program (SCHIP). CMS is the largest purchaser of health care in the United States — our programs account for more than one third of the dollars spent on health care in the U.S. economy. In 2001, CMS programs provided health care coverage for 70 million people — nearly one out of every four Americans.

In administering the Medicare, Medicaid and SCHIP, CMS activities are performed with one thing in mind: a commitment to the people who rely upon our programs for access to and quality of the health care they need. CMS' mission, though complex in execution, is simply stated: We assure health care security for beneficiaries. And our vision is no less ambitious: In the stewardship of our programs, we lead the Nation's health care system toward improved health for all.

CMS could not accomplish its mission by working alone. We work in partnership with many other organizations and individuals: other Federal and State agencies, and Territorial and Tribal governments; private-sector contractors; health plans, practitioners, and health care facilities; professional organizations, academia, and advocacy groups; and beneficiaries and their families.

II. Scope and Applicability of Guidelines

CMS will ensure that disseminated information meets the standards of quality set forth in the OMB, HHS and CMS guidelines. It is CMS's goal to ensure and maximize the quality, objectivity, utility, and integrity of information that it disseminates to the public. We strive to provide information that is accurate, reliable, clear, complete, unbiased, and useful. We are committed to integrating the principle of information quality into every phase of information development, including creation, collection, maintenance, and dissemination.

The pre-dissemination review described in the guidelines only applies to substantive information disseminated on or after October 1, 2002. The administrative mechanism for correction applies to information that the agency disseminates on or after October 1, 2002, regardless of when the agency first disseminated the information.
 

1. Information Covered Information that IS subject to the Information Quality Guidelines includes:

   • statistics and information designed for public dissemination to help CMS program beneficiaries make informed choices
   • statistical or actuarial information
   • studies and summaries prepared for public dissemination to inform the public about the impact of CMS programs
   • studies and summaries prepared for use in formulating broad program policy.

   
   To correct information that is subject to the Information Quality Guidelines, please follow the procedure for correcting CMS information outlined in Section VI of this notice.
    

2. Information Not Covered Information that is NOT subject to the Information Quality Guidelines includes:
   • personal information maintained for purposes of enrollment or adjudicative decisions
   • data used to verify and pay individual claims
   • information pertaining to basic agency operations, such as the Chief Financial Officer's Report
   • procedural and policy manuals
   • management information that is produced primarily for internal use
   • information which is disseminated only to CMS employees or agency contractors
   • opinions where the agency's presentation makes it clear that what is being offered is personal opinion rather than fact or the agency views
   • press releases that support the announcement or give public notice of information that CMS has disseminated elsewhere.

III. Types of Information Disseminated

Generally speaking, information released by CMS falls into one of two broad categories: Medicare consumer information, and program information and evaluation. Typically, CMS does not provide detailed consumer information about Medicaid or SCHIP because these functions are carried out by individual State governments.
 

1. Medicare consumer information This information is designed to help people enrolled in Medicare make informed decisions about the types of insurance and service providers they use. For example,
   • Medicare & You is an annual handbook mailed to Medicare enrollees. The book provides basic benefit information, and also provides information about managed care plans and other types of health insurance available to enrollees.
   • Nursing Home Compare is a web site containing information about long-term care facilities. In combination with other factors, this information can be used by people who are contemplating nursing home care for themselves or others, to decide which facility is best for them.
2. Program information and evaluation This type of information is designed to help improve the performance of CMS programs. Examples include:
   • Program information provides summary statistics describing use of CMS program services, the number of people using those services, expenditure for those services, etc. For example, CMS publishes an annual Medicare and Medicaid Statistical Supplement to the Health Care Financing Review.
   • Statistical data sets are claims-level, person-level, or provider-level data used for program evaluation, health services research, or policy development. For example, CMS prepares Standard Analytic Files (SAFs) extracting Medicare hospital inpatient claims that can be used by outside researchers who abide by a data use agreement. The agency also releases information from the Medicare Current Beneficiary Survey (MCBS), a survey of a random sample of Medicare enrollees that collects social, demographic, and economic information about those enrollees.
   • Research and evaluation reports focus on issues germane to CMS programs and policy. Typically, each innovation in CMS programs must be evaluated; often, such innovations are preceded by research performed by CMS staff or by outside researchers under contract to the agency. Many of these evaluations are released through Reports to the Congress.
   • Technical reports summarize specific aspects of CMS and its program activities. For example, each year CMS actuaries produce the data underlying the report of Medicare's trustees on the financial health of the program.
   • Payment updates establish the amount Medicare will pay for particular services or for capitated care of beneficiaries. For example, each year the agency publishes a fee schedule update that determines payments for physician services.

IV. Types of Dissemination Methods

CMS uses four modes to disseminate information.
 

1. Internet The agency maintains two web sites. One of them — www.medicare.gov — is devoted to consumer information. It is intended for use by program beneficiaries, their families and care-givers, and their advocates. The second site — www.cms.hhs.gov — is designed to serve researchers, policymakers, and health care providers. The sites are cross-linked for ease of access.
    
2. Printed matter CMS produces a wide variety of pamphlets, handbooks, and directories related to our programs. For example, CMS publishes the Health Care Financing Review, containing scholarly articles relevant to CMS programs as well as data highlights from those programs. In addition to the Medicare & You handbook described above, smaller leaflets are prepared to describe program eligibility, enrollment, and benefits.
    
3. Storage Media Data from CMS operations are available in a variety of storage media upon request for use at non-CMS computing facilities. Aside from download files available through the Internet, this form of information dissemination is ad hoc in nature.
    
4. Telephone CMS maintains a national toll-free telephone number, 1-800-MEDICARE, to process questions and provide information about the Medicare program.
    
5. Multi-media CMS is engaged in an ongoing multimedia effort to provide information about the Medicare program to its enrollees. The National Medicare Education Program (NMEP) is unique among the agency's information products in that it is intentionally designed to incorporate Internet, printed, and telephone information seamlessly.

V. Agency Quality Assurance Policies, Standards and Processes

At CMS, the quality assurance process begins at the inception of the information development process. Information released by CMS is developed from reliable data sources using accepted methods for data collection and analysis, and is based on thoroughly reviewed analyses and models. Further, CMS reviews the quality (including the objectivity, utility, and integrity) of information before it is disseminated and treats information quality as integral to every step of the development of information, including its creation, collection, maintenance and dissemination.  The guidelines below describe procedures that CMS employs to assure the quality of its information products, including their utility, objectivity, integrity, transparency, and reproducibility.

1. Utility Utility involves the usefulness of the information to its intended users. Utility is achieved by staying informed of information needs and developing new data, models, and information products where appropriate. Based on internal analyses of information requirements, convening and attending conferences, working with advisory committees, and sponsoring outreach activities, CMS keeps abreast of information needs with respect to the analysis of CMS programs. To the extent possible, we base our analysis on CMS administrative data and surveys by other federal agencies or established survey organizations. When major needs for data related to CMS populations are identified, CMS conducts special-purpose surveys to address these needs. CMS's ongoing publication series and other information products are reviewed to ensure that they remain relevant and address current information needs. Based on internal product reviews, consultation with users, and in response to changing needs and emphases, content of ongoing information products is changed, new products are introduced and others discontinued. CMS prepares special reports and topical studies that address emerging information needs stemming from proposed changes in the law and related policy debates. Where appropriate, contact information is available on each publication (and in some cases on each table of a publication) to allow feedback and questions by users. New and revised information products are tested with focus groups of intended recipients. In many cases, the structure of the content itself is a collaborative process involving providers, consumers, academicians, and policy analysts.
    
2. Objectivity Objectivity involves a focus on ensuring that information is accurate, reliable and unbiased and that information products are presented in an accurate, clear, complete and unbiased manner. Objectivity is achieved by using reliable data sources and sound analytical techniques, and carefully reviewing information products prepared by qualified people using proven methods. Using reliable data sources. Much of the information disseminated by CMS is based on administrative data files. These files contain information used to manage CMS programs, including data to determine benefit entitlement, and to compute and pay benefits. CMS staff conduct ongoing reviews of claims information in CMS data systems to ensure its accuracy. CMS administrative data are also covered under CMS's Financial Management Systems and conform to the high standards of financial accountability demanded by these Systems. These financial management systems are mandated by the Office of Management and Budget and are designed to provide complete, reliable, consistent, timely and useful management information to enable agencies to carry out their fiduciary responsibilities. CMS-sponsored surveys are conducted using methodologies that are consistent with generally accepted professional standards for all aspects of survey development, including sample frame development, statistical design of the survey sample, questionnaire design and testing, data collection, sampling and coverage errors, nonresponse analysis, imputation of missing data, weights and variance estimates. CMS surveys follow guidelines and policies set forth in the Paperwork Reduction Act (PRA) and other regulations related to the conduct of government surveys. Through the PRA process, CMS ensures that information will be collected, maintained, and used in a way that is consistent with OMB, HHS and CMS information quality guidelines. CMS also prepares information products using data produced or maintained by other federal agencies and established survey organizations. Where possible, external data used by CMS are produced using generally accepted methodologies. Where samples from administrative data files are employed for analysis, sound statistical methods are employed to develop samples. Staff involved in producing statistical publications are knowledgeable about the content, structure and limitations of the administrative data files and maintain working relations with staff who create, update and maintain these files. Where administrative files are linked to surveys for analysis, sound procedures for extracting and linking data from external sources are employed, based on a thorough understanding of the relevant components of the data sources. Computer programs for sampling, extracting and linking external data files are developed and tested by technically qualified staff. Samples from administrative files are evaluated to ensure that samples are representative of the underlying administrative data files. Estimates prepared from external data sources are reviewed to ensure that the data extraction and linkage processes were implemented correctly. Using sound analytic techniques. Analytical reports are prepared using a variety of analytical techniques. These techniques range from simple tabulations and descriptive summary statistics to multivariate statistical methods and econometric models. Analytical techniques are reviewed for their appropriateness to the data and the analysis being conducted and are clearly identified in reports. Actuarial estimates are subject to the standards of that profession and are certified by the Chief Actuary. Preparing statistical data products. Estimates in statistical data products are prepared from representative random samples of CMS administrative data files and from reliable external data sources. Procedures for sampling from administrative files and linking external data files to administrative data files are prepared using accepted statistical methods. Output is reviewed by knowledgeable staff within the originating component and across components as appropriate. Estimates are compared to prior year estimates and estimates from other sources to ensure consistency, reasonableness and reliability. All data sources used in producing statistical data products are identified, either for the publication as a whole or for individual tables. Documentation includes specification of variables used, definitions of variables when appropriate, sampling errors and disclosure avoidance rules or techniques. Preparing analytical reports and policy studies. Information contained in analytical reports and policy studies is based on estimates derived from reliable administrative data files and external data sources. Analysts apply sound statistical and analytical techniques and are knowledgeable about the data sources and models being used. All data sources are identified. When analyses are based on simulation model projections, the assumptions used to produce the projections are also identified as well as the rationale for the assumptions used and the impact of using alternative assumptions. All analytic reports and policy studies are reviewed by technically qualified staff to ensure that analysis is valid, complete, unbiased, objective and relevant. Analytic reports and policy studies that are considered to be more technically complex are also reviewed by subject matter experts outside of the originating component to provide additional perspective and expertise. Establishing editorial review for accuracy and clarity of information in publications. All information products are edited and proofread before release to ensure clarity and coherence of the final report. Text is edited to ensure that the report is easy to read and grammatically correct, that thoughts and arguments flow logically, and that information is worded concisely and lucidly. Tables and charts are edited to ensure that they clearly and accurately illustrate and support points made in the text, and include concise but descriptive titles. Tables and charts clearly indicate the unit of measure and the universe being examined and all internal labels (column heads, row stubs, and panel headings) accurately describe the information they contain. All changes made to a manuscript during the editing process are checked by a proofreader and reviewed and approved by the author.
    
3. Integrity Integrity refers to the security of information from unauthorized access or revision to ensure that the information is not compromised through corruption or falsification. CMS has in place rigorous controls to ensure the integrity of its administrative information. CMS is highly protective of the confidentiality of information it holds through its policies and practices. CMS administers social insurance programs that — directly or indirectly — touch the lives of almost every American. CMS has in place programs and policies for securing CMS resources as required by the Government Information Security Reform Act (P.L.106-398, Title X, Subtitle G). These security procedures address all major components of information security. CMS is subject to a number of statutory requirements that protect the sensitive information it gathers and maintains on individuals. Among these are:
   • Health Insurance Portability and Accountability Act of 1996
   • Privacy Act of 1974
   • Computer Security Act of 1987
   • Office of Management and Budget (OMB) Circulars A-123, A-127, and A-130
   • Government Information Security Reform Act
   • Federal Managers' Financial Integrity Act (FMFIA) of 1982
4. Transparency and Reproducibility If an agency is responsible for disseminating "influential" information, guidelines for dissemination should include a high degree of transparency about data and methods to facilitate its reproducibility by qualified third parties. Information is considered influential if it will have a substantial impact on important public policies or important private sector decisions. CMS's guidelines call for identification and documentation of data sets used in producing estimates and projections, and for clear descriptions of the methods used. Many estimates and projections included in CMS information products are not directly reproducible by the public because the underlying data sets used to produce them are confidential. However, some statistical publications that are based on publicly available data and whose programs are made available on request are fully reproducible by the public. Further, some CMS data is available to people outside the agency through adherence to a data use agreement, provided that the use of the data is consistent with the appropriate systems of records notice. Where estimates and projections may not be easily reproduced by third parties due to the complexity and detail of the methods and data, greater emphasis is placed on periodic review by outside panels of technical experts. CMS also achieves transparency through wide dissemination of its information. Many reports and other data products are available both as printed and electronic documents. New documents posted on our Web sites are section 508 compliant, making information available to an audience that includes persons who have a visual impairment and read online using assistive technology. (Older documents are being converted to meet section 508 standards.)

VI. Agency Administrative Complaint Procedures

CMS has developed administrative mechanisms to allow affected persons to seek and obtain correction of disseminated information that does not comply with OMB, HHS and CMS guidelines.

Existing public comment procedures for rule-makings and other formal agency actions already provide well established procedural safeguards that allow affected persons to raise information quality issues on a timely basis. Accordingly, CMS will use these existing procedures to respond to information quality complaints that arise in this process.

In cases where agency disseminates a study, analysis, or other information prior to the final agency action or information product, requests for correction will be considered prior to the final agency action or information product in those cases where in the agency's judgment issuing an earlier response would not unduly delay issuance of the agency action or information product and the complainant has shown a reasonable likelihood of suffering actual harm from the agency's dissemination if the agency does not resolve the complaint prior to the final agency action or information product.
 

1. Responsibility of the Complainant To seek a correction of information disseminated by the agency, individuals shall follow the procedures described below.

   1. A complaint or request for review and correction of information shall be in written hard copy or electronic form;
   2. it shall be sent to the agency by mail or electronic-mail (e-mail) at the address below; and
   3. it shall state that an information quality request for correction is being submitted.

   
   The complaint shall contain

   4. a detailed description of the specific material that needs to be corrected including where the material is located, i.e. the publication title, date, and publication number, if any, or the web site and web page address (url), or the speech title, presenter, date and place of delivery;
   5. the specific reasons for believing the information does not comply with OMB, HHS, or CMS guidelines and is in error and supporting documentation, if any;
   6. the specific recommendations for correcting the information;
   7. a description of how the person submitting the complaint is affected by the information error; and
   8. the name, mailing address, telephone number, e-mail address, and organizational affiliation, if any, of the individual making the complaint.

   
   Complainants should be aware that they bear the "burden of proof" with respect to the necessity for correction as well as with respect to the type of correction they seek.
    

2. Responsibility of the Agency Based on a review of the information provided, the agency will determine whether a correction is warranted and if, so what action to take. The agency will respond to the requestor by letter or e-mail. The agency's response will explain the findings of the review and the actions that the agency will take, if any. The response will consider the nature and timeliness of the information involved and such factors as the significance of the correction on the use of the information and the magnitude of the correction. The response will describe how the complainant may request reconsideration. The agency will respond to all requests for correction within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.
    
3. Appeals If the individual submitting the complaint does not agree with the agency's decision (including the corrective action, if any), the complainant may send a written hard copy or electronic request for reconsideration within 30 days of receipt of the agency's decision. The appeal shall state the reasons why the agency response is insufficient or inadequate. Complainants shall attach a copy of their original request and the agency response to it, clearly mark the appeal with the words, "Information Quality Appeal," and send the appeal to the specific agency appeals address. The agency official who resolved the original complaint will not have responsibility for the appeal. The agency will respond to all requests for appeals within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.
    

4. Contact Complaints filed under this guideline shall be mailed to:

   • Information Quality
     Centers for Medicare & Medicaid Services
     7500 Security Boulevard
     Baltimore, MD 21244-1850

   
   Alternatively, the complaint can be emailed to infoquality@cms.hhs.gov Appeals filed under this guideline should be mailed or emailed to the same address, and should contain the word "appeal" on the envelope or in the subject line.

VII. Influential Scientific, Financial, and Statistical Information

The OMB Information Quality Guidelines require that "influential" scientific, financial, or statistical information in official Government documents must be based on data or studies that could be substantially reproduced if the original or supporting data were to be independently reanalyzed using the same methods. "Influential" means that CMS can reasonably determine that dissemination of the information will have a substantial impact on important public policies or important private sector decisions or will have important consequences for specific health practices, technologies, substances, produces, or firms. Examples of the types of information disseminated by CMS that have the potential to be influential include the annual reports of the Medicare Board of Trustees and annual publication of provider payment rates.

CMS is committed to applying rigorous scientific standards to ensure the accuracy and reliability of program evaluation results. The scientific/research, financial, and statistical community recognizes peer review as the primary means of quality control. CMS routinely seeks input from qualified peer reviewers, inside and outside the Federal government prior to dissemination of this type of information.

This guidance provides the information quality guidelines for the Food and Drug Administration (FDA) requested under the Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies (OMB Guideline). ¹ The OMB Guidelines implement section 515 of the Treasury and General Government Appropriations Act for Fiscal Year 2001 (Pub. L. 106-554; H.R. 5658). The OMB Guidelines direct agencies to issue their own information quality guidelines ensuring and maximizing the quality, objectivity, utility, and integrity of information, including statistical information, disseminated by the agency. This guidance is part of the U.S. Department of Health and Human Services (HHS) implementation plan developed to comply with the OMB Guidelines. The principles described in the HHS plan and OMB guidance are incorporated by reference into this guidance.

This guidance describes the nature of the information disseminated by the FDA and explains FDA's standards, policies, and procedures for ensuring the quality of the information it disseminates. The guidance also explains the administrative mechanisms that are in place to enable persons to seek and obtain correction of information maintained and disseminated by the FDA that they believe does not comply with the OMB and HHS Guidelines and this guidance.

I. Agency Mission

The Agency's mission, as defined in Section 406 of the Food and Drug Administration (FDA) Modernization Act of 1997, is as follows:

The Administration shall --

1. promote the public health by promptly and efficiently reviewing clinical research and taking appropriate action on the marketing of regulated products in a timely manner;
    
2. with respect to such products, protect the public health by ensuring that --
   •  
     1. foods are safe, wholesome, sanitary, and properly labeled;
     2. human and veterinary drugs are safe and effective;
     3. there is reasonable assurance of the safety and effectiveness of devices intended for human use;
     4. cosmetics are safe and properly labeled;
     5. public health and safety are protected from electronic product radiation; and
     6. when the Family Smoking Prevention and Tobacco Control Act became law on June 22, 2009, it gave FDA the authority to regulate tobacco products.
3. participate through appropriate processes with representatives of other countries to reduce the burden of regulation, harmonize regulatory requirements, and achieve appropriate reciprocal arrangements; and
    
4. as determined to be appropriate by the Secretary, carry out paragraphs (1) through (3) in consultation with experts in science, medicine, and public health, and in cooperation with consumers, users, manufacturers, importers, packers, distributors, and retailers of regulated products.

II. Scope and Applicability of Guidelines

FDA is a scientific regulatory agency that regulates one trillion dollars worth of consumer goods each year. This amounts to more than 20 percent of all consumer spending. FDA regulates most food that we eat, all prescription drug and over-the-counter drug products that we take, and all medical devices that we use. For foods, we ensure that products are safe, wholesome, sanitary, and properly labeled. For drugs, we ensure the products are both safe and effective for use; and for medical devices, there is a reasonable assurance of their safety and effectiveness. We also are responsible for ensuring that electronic and radiation-emitting equipment, such as x-ray machines, microwave ovens, and metal detectors, are safe for use. We certify and inspect annually all mammography facilities. We regulate animal feed and most animal drugs. We ensure that cosmetics are labeled honestly and cause no harm. Our regulatory activities include inspection and surveillance of marketed products, standard setting, bioresearch monitoring, and human subject protection. We also conduct research in support of our regulatory programs.

FDA, which employs about 10,000 people, is organized primarily by centers:

• Center for Biologics Evaluation and Research (CBER)
• Center for Devices and Radiological Health (CDRH)
• Center for Drug Evaluation and Research (CDER)
• Center for Food Safety and Applied Nutrition (CFSAN)
• Center for Veterinary Medicine (CVM)
• National Center for Toxicological Research (NCTR)
• Office of the Commissioner (OC)
• Office of Regulatory Affairs (ORA)
• Center for Tobacco Products (CTP)

Our Office of the Ombudsman, which is responsible for monitoring complaints regarding information dissemination and our response to those complaints, resides in the Office of the Commissioner.

As described in detail in the paragraphs that follow, we disseminate many different types of information to a wide variety of audiences, including the regulated industry, health care professionals and organizations, consumers, patients, other governmental agencies, and international organizations and agencies. Information dissemination is an important part of our mission to promote and protect the public health. FDA recognizes that public access to high quality information is critical to achieving this mission and public input, in turn, improves the quality of the information we disseminate.

Because of the nature of this information, our goal has been and remains to ensure that all the information we disseminate meets the high standards of quality (including objectivity, utility, and integrity) described in the OMB and HHS Guidelines. As discussed in detail in Section V, we have established several policies, standards, and processes to ensure the quality of the information we make available to the public. FDA also will ensure that information collected under the Paperwork Reduction Act of 1995, 44 U.S.C. chapter 35, will be collected, maintained, and used in a manner consistent with the OMB and HHS Guidelines and this guidance document. With the following specific exceptions, the OMB Guidelines apply to most categories of FDA-disseminated information (see discussion in following sections):

1. Documents relating to internal FDA procedures
2. Internal government correspondence
3. Correspondence with individuals that is not normally made public
4. Press releases (unless they contain new substantive information not covered by previous information dissemination)
5. Archival records
6. Distributions intended to be limited to subpoenas or adjudicative documents including findings and determinations made in the course of an administrative proceeding pursuant to 21 CFR Parts 10, 12, 13, 14, 15, 16, 17 and 19.
7. Scientific publications that only contain the views of the authors and are not used to support an official agency position
8. Responses to requests for information under the Freedom of Information Act (FOIA), the Privacy Act, the Federal Advisory Committee Act, or other similar laws.

In the pages that follow, we describe the types of information we disseminate (Section III), the methods we use to disseminate this information (Section IV), and the quality assurance policies, standards, and processes that have been put in place to ensure the quality of the information we distribute (Section V). Some of the information described below may include information that falls under one of the types of information specifically excluded above. To the extent that information in one of the categories listed below includes information listed in one of the exceptions, the OMB Guidelines do not apply. In Section VI, we discuss the procedures to be used by persons who want to request that information we have disseminated be corrected. Section VII discusses the types of information we have identified to be influential according to the OMB Guidelines and the principles we apply to information that will be disseminated regarding risks to human health, safety, and the environment. The principles that are used have been adapted from the quality principles applied by Congress to risk information pursuant to the Safe Drinking Water Act Amendments of 1996 (42 U.S.C. 300g-1(b)(3)(A) and (B)). Section VIII describes some special circumstances that may apply to certain information dissemination activities.

The pre-dissemination review described in the HHS guidelines only applies to information disseminated on or after October 1, 2002. The administrative mechanism for correction applies to information that the agency disseminates on or after October 1, 2002, regardless of when the agency first disseminated the information.

III. Types of Information Disseminated

We make a large number of documents and information available to a variety of audiences. The major types, with examples of each, are provided here.

1. Public Communications About Risk

   As part of our mission to protect the public health and safety, we provide the public with a wide variety of information on risk, including information on food safety and the risks involved with using medical products. Some examples of our communications are listed here.

   • Consumer advice and fact sheets (for example, News Release No. 0020.01, January 2001 provides advice to consumers to reduce the risk of illness from foodborne Listeria monocytogenes)
   • "Dear Health Care Professional" letters (for example, Agency August 2001 letter to health care professionals warning about rhabdomyolsis, a serious, potentially fatal adverse effect of all statin drugs)
   • Public health and safety alerts (for example, "FDA Announces New Alcohol Warnings for Pain Relievers and Fever Reducers," October 1998; "Seniors and Food Safety -- Preventing Foodborne Illness," May 1999; "Learning About Lasik Eye Surgery," on the Internet; "FDA Patient Safety News," a video targeting hospitals; "Tips and Warnings for Consumers About Buying Medicines and Medical Products Online," on the Internet; "An Important Message for Pregnant Women and Women of Childbearing Age About the Risk of Mercury in Fish," March 2001; "FDA Public Health Notification: Reducing Radiation Risk from Computed Tomography for Pediatric and Small Adult Patients," November 2001)
   • FDA Talk Papers (for example, FDA Announces Availability of Vibrioparahaemolyticus Risk Assessment, January 18, 2001)
   • Various subject matter brochures intended for consumers. Some are produced as low-literacy brochures aimed at consumers with no more than a fifth-grade reading level. Some are also produced in other languages, particularly in Spanish. Some are mailed directly to specific audiences; some are disseminated at large professional meetings (for example, "Reprocessing of Single-Use Medical Devices by Hospitals" (November 2000), "Buying Contact Lenses on the Internet, by Phone, or by Mail" (August 2001)).
   • Posters are distributed to health clinics and schools with consumer information on FDA-related health issues.

2. Rulemaking Documents

   Like many Federal agencies, we engage in rulemaking. This process includes publishing a proposed rule and explanatory material in the Federal Register, obtaining public comment, and publishing a final rule and response to the comments. Some examples include:

   • Additional Criteria for Classifying Over the Counter Drugs as Generally Recognized as Safe and Effective and Not Misbranded, a final rule that published on January 23, 2002
   • Medical Devices; Device Tracking, a final rule that published on February 8, 2002
   • State Certification of Mammography Facilities, a final rule that published on February 6, 2002
   • Implantation or Injectable Dosage Form New Animal Drugs: Trenbolone and Estradiol; a final rule that published on February 7, 2002
   • Foreign Establishment Registration and Listing, a final rule that published on November 27, 2001
   • Substances Affirmed as Generally Recognized as Safe: Menhaden Oil; a proposed rule that published on February 26, 2002
   • Hazard Analysis and Critical Control Point (HACCP): Procedures for the Safe and Sanitary Processing and Importing of Juice; a final rule that published on January 19, 2001

   As part of the rulemaking process, we may also publish advanced notice of proposed rulemaking documents (ANPRs) and direct final rules.

3. Product Approvals

   1. Medical Products

      When we evaluate applications for approval to market medical products, we produce reviews of the data collected, analyzed, and submitted by applicants. On approval of drugs², we compile our reviews into an approval package that provides the basis for the Agency clearance of a decision to approve a product. To provide as much information as possible to health care practitioners and consumers so they can make informed decisions about treatment, we make the medical product approval packages, including generic drug approvals, available on the Internet. An approval package can range from 100 to more than 1,000 pages, redacted to remove confidential and trade secret information. Contents usually include individual discipline reviews; correspondence between the company and FDA; administrative documents; and labeling. For example, the package for Clarinex [desloratadine], an antihistamine to treat seasonal allergic rhinitis, was approved on December 21, 2001. We posted the 500-page package on the Internet on February 12, 2002.

      We also post on the Internet, for animal drug products, an FOI summary of the approval and for device premarket approvals, a detailed summary of safety and effectiveness, the approval order, and the draft labeling.

   2. Food and Color Additives

      When we evaluate applications for premarket approval of food additives and color additives, we produce reviews of the data and analyses submitted by the applicant. For direct food additives and color additives, we publish a final rule in the Federal Register explaining the basis for Agency approval of the product and issuance of a regulation in the U.S. Code of Federal Regulations. For other food additives, we announce their approval via Internet listings that are updated at least monthly. In all cases, the published listings include specifications and use limitations necessary to ensure the safe use of the product. In addition, the documents and information that form the basis for Agency approvals are available under the Freedom of Information Act.

4. Guidance and Regulatory Assistance

   We develop guidance and other policy documents usually with input from the public, to assist industry, consumers, hospitals, reviewers, and other health care related organizations and individuals interested in our statutes and regulations. In addition, we provide procedural guidance to our field offices. In compliance with our policy involving good guidance practices, under 21 CFR 10.115, we publish notices announcing the availability of guidances in the Federal Register, and make the guidances available in the public docket and on the Internet. Some examples of guidances are provided here:

   • Guidance to Hospitals, Nursing Homes, and Other Health Care Organizations -- FDA Public Advisory on the risks of death and injury related to medical gas mix-ups (April 2001)
   • Small Business Compliance Guides -- guidances to help small businesses; required by the Regulatory Flexibility Act (5 U.S.C. 602) for all rulemakings that will have a significant impact on small entities (for example, Sterility Requirement for Aqueous-Based Drug Products for Oral Inhalation -- Small Entity Compliance Guide, November 2001)
   • Formal Dispute Resolution: Appeals Above the Division Level (March 2001)
   • Food Security Guidance (January 2002)
   • Food Code; 2001 Revision (December 2001)
   • Guidance for Industry: Studies to Evaluate the Safety of Residues of Veterinary Drugs in Human Food: Genotoxicity Testing, VICH GL23
   • Guidance on Enforcement Priorities for Single-Use Devices Reprocessed by Third Parties and Hospitals (August 2000)
   • The Policy Guidance Help System (January 2000; revised November 2001) -- a computerized system containing all Mammography Quality Standards Act regulations and final guidance (Internet and as a stand-alone downloadable program)
   • CDRH develops "Plain Talk" guidance on how to comply with our regulations and provides a unique interactive Website called "Device Advice" to answer specific device related questions.
   • Compliance Policy Guides are issued to the field offices to ensure that our regulations and policies are followed consistently. These Compliance Policy Guides are made public and treated as guidances.
   • Regulatory Procedures Manual provides FDA procedural guidance and instruction for use by FDA and the public. The manual is publicly available and is treated as guidance.

5. Reports

   We also develop reports on a variety of topics. Some examples include:

   • Managing the Risks From Medical Product Use -- Report to the FDA Commissioner from the Task Force on Risk Management (May 1999)
   • Prescription Drug Marketing Act of 1987 -- Report to Congress (June 2001)
   • FDA Fiscal Year 2002 Congressional Budget Request (annual report)
   • Import Detentions Reports (IDRs) -- IDRs provide information on the products detained by the Agency (that is, products for which our District Offices have issued a "Notice of Detention and Hearing"). The IDR is generated from data collected by FDA's Operational and Administrative System for Import Support (OASIS) and is updated monthly and is posted on the FDA website for a period of time (on the average of 12 months). Archived data are not available on the Internet but is available under the Freedom of Information Act.
   • FDA Enforcement Report -- Published weekly, this online publication contains information on recalls and other actions taken in connection with Agency regulatory activities.
   • Mammography Facility Adverse Event Report -- an annual report of adverse actions taken against mammography facilities issued to help health professionals and consumers in evaluating the performance of their mammography facilities

6. Citizen Petitions and Responses

   When citizens petition the FDA to address an issue, we write a response to the petitioner explaining our position. Although these responses are letters addressed to individuals or organizations, the petitions and our responses are made available through the public docket and often on the Internet. In recent years, we have responded to an average of 250 citizen petitions per year. For example, on February 15, 2002, we issued a response to a petitioner asking us to refrain from approving a generic version of an antibiotic (Ceftin) if the generic drug product's active ingredient were wholly or partially in crystalline form.

7. Press Items and Publications

   The Agency releases much information through the press and related media. Some examples include:

   • News releases (for example, HHS and USDA Release Listeria monocytogenes Risk Assessment and Listeria monocytogenes Action Plan, Release No. 0020.01, January 18, 2001; Cape May Foods Recalls Chopped Clams Because of Possible Health Risk, January 15, 2002)
   • FDA Consumer Magazine (bimonthly magazine targeting consumers) and reprints of selected feature articles from FDA Consumer.
   • FDA Veterinarian (bimonthly newsletter targeting veterinarians and the food animal industry)
   • Frequently Asked Questions: What Can I Do to Protect Myself from Food Poisoning? (Internet only)

IV. Types of Dissemination Methods

Transparency is one of the Agency's key goals. It is critical that our audience understand what we do, how we do what we do, and why we do something. Because our audience is so diverse, we use a variety of media to disseminate public health and safety information. Some examples are provided here:

• Oral Presentations in public forums sponsored by FDA or outside parties, such as professional societies or trade associations (for example, the FDA Science Forum)
• Internet (medical product approval packages; device summaries of safety and effectiveness; safety alerts; guidance documents; special issue papers, such as those on "Online Medicine Sales" and the "Agency's Bioterrorism Activities")
• Federal Register (proposed and final rules; notices announcing the availability of guidances; meeting notices; other notices)
• The public docket (citizen petitions and responses; transcripts of certain meetings; information about advisory committee meetings; public comments on guidances, regulations, and any other documents that publish in the Federal Register)
• Videos (on the Internet and for dissemination to outside organizations and use in meetings and conferences. For example, "Science and our Food Supply: Investigating Food Safety from Farm to Table")
• CD ROM (for example, Listeria monocytogenes draft risk assessment documents and models)
• FDA's E-mail lists (see topics at www.fda.gov/emaillist.html)

V. Agency Quality Assurance Policies, Standards, and Processes

As described in the HHS Implementation Plan, as one of the HHS operating divisions, we have established a number of quality assurance policies, standards, and processes for ensuring the quality of the information we disseminate to the public. Our documents undergo a rigorous review and clearance evaluation according to pre-established procedures, documented in our regulations and guidances.

Generally, Agency documents are cleared as follows:

• Document is developed by an individual or team
• Document is circulated to working group members, and often an editor, and comments are incorporated
• Document is circulated and cleared by center managers
• We publish many documents (for example, guidances, proposed rules) for comment by members of the public, and some documents are reviewed by outside advisory committees comprising experts in the subject matter of the document
• If required by regulation or policy, documents are circulated to and cleared by the Office of the Chief Counsel, Office of Policy, Planning, and Legislation, the Department (HHS), and the Office of Management and Budget (OMB).

In addition to these clearance procedures, we use a number of mechanisms to ensure the quality of the information we disseminate. FDA reviews the quality (including the objectivity, utility, and integrity) of information before it is disseminated and treats information quality as integral to every step of the development of information, including its creation, collection, maintenance, and dissemination. Quality, as defined in the OMB Guidelines, encompasses (1) utility, the usefulness of the information to its intended users, including the public; (2) objectivity, whether information is being presented in an accurate, clear, complete, and unbiased manner; and (3) integrity, the information is protected from unauthorized access or revision.

1. Utility

   We only disseminate information that we believe will be useful to the public or a segment of the public. In fact, often we disseminate information because members of the public or the regulated industry have requested it. We develop many guidances as a result of public questions about a specific topic. We also have processes (21 CFR 10.30) by which members of the public can petition us to take certain actions, such as initiating rulemaking or taking specific administrative or enforcement actions. Requests for dissemination of information also can be submitted through petitions.

   We developed our good guidance practice (GGP) policy as a result of public request. Congress later enacted the policy into law, and we codified our GGP policy in our regulations at 21 CFR 10.115. The GGPs describe our procedures for developing, issuing, and using guidance documents and include detailed procedures on how members of the public can suggest areas for guidance development, submit drafts of proposed guidance documents, and request the revision or withdrawal of an existing guidance document. We also maintain a guidance Agenda, which is a list of guidances we are planning to develop in the coming year. We post the list on the Internet and publish it annually in the Federal Register. We publish the Agenda to keep the public up-to-date on guidance development plans and solicit input from the public on what guidances are needed.

   In addition, we are subject to the Freedom of Information Act and the Electronic Freedom of Information Act Amendments (5 U.S.C. 552), which provide for the dissemination of information to members of the public and posting on the Internet certain information that is, or is likely to be, responsive to multiple information requests.

   In accordance with the Regulatory Flexibility Act (5 U.S.C. 602), the General Services Administration publishes a semiannual regulatory agenda describing the regulatory actions being developed. The Secretary welcomes comments on this agenda and suggestions for improvements and initiatives.

2. Objectivity

   As already mentioned, we have many different systems in place to ensure that the information we disseminate is presented in an accurate, clear, and unbiased manner. We have a strong commitment to writing all our new documents in plain English. We have provided plain English training to many of the employees who write our documents. We also continue to solicit feedback from stakeholders on our efforts to present written information clearly.

   We also take steps to ensure that our regulatory decisions are based on objective information. If we rely on information submitted to us by third parties in support of an application for product approval or in a rulemaking proceeding, we make sure that this information meets the appropriate standards for quality and objectivity. We have a number of regulations and guidances that set standards for the generation of information in support of regulatory decisions. For example, the supporting data may be generated in new research using good laboratory practices (GLPs) ³, in clinical studies subject to Good Clinical Practices (GCPs) ⁴, in reviews of existing information obtained primarily from peer-reviewed scientific literature, or obtained from surveys based on widely accepted scientific survey techniques. Interpretations of quantitative results of Agency studies are commonly subjected to statistical analyses. Even though we may be unable to fully disclose the information on which our actions are based because it has been submitted by a third party with a proprietary interest in maintaining its confidentiality, we make special efforts to ensure that the information submitted meets these standards for objectivity and is of high quality.

   The methods by which we ensure the objectivity of the information for some of our major regulatory activities are described here.

   1. Product Review Activities

      One key FDA responsibility is the evaluation of data submitted to the Agency in medical and veterinary medical product applications and in food and color additive petitions or notifications. In general, firms that want to market certain products (for example, drugs and medical devices) submit applications to FDA. These applications contain data or information on which the firm relies to claim that its product is safe and effective for its intended uses. We base our decisions about safety and effectiveness primarily on our analyses of the integrity of the submitted data. When we approve a product and post a drug review package on the Internet, we are ensuring that our analyses of the data submitted to us are available for public scrutiny.

      We develop regulations and guidance documents to help ensure that the data submitted to us result from the best available studies, that the studies are conducted in accordance with sound and objective scientific practices, and that the data are collected using scientifically accepted methods. For example, FDA regulations specify the format and content of the clinical studies that are submitted in support of an application to market a new drug product. They specify how the data are to be collected and the types of analyses that are to be performed. In the case of biological products, we have developed guidance on the format and content of reports on clinical studies that are submitted to the Agency. Other FDA guidances provide detailed descriptions of appropriate methodologies, analyses, and procedures.

      Since the early 1990s, we have been involved in an intensive international effort to harmonize technical requirements for the conduct of studies in support of marketing applications and the content and format of applications with the goal of allowing the submission of a common application for marketing around the world. The International Conference on Harmonisation for Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH) brings together scientific experts from different countries to develop a consensus on the appropriate requirements. We also are engaged in international activities in the device, food, and animal drug areas. For example, International Cooperation on Harmonisation of Technical Requirements for Registration of Veterinary Products (VICH) is the veterinary counterpart to ICH. The Global Harmonization Task Force (GHTF) is working to harmonize device regulations and guidance. The Codex Alimentarius committees are working to harmonize international food regulations. Many agreements reached are then embodied in regulations issued through notice-and-comment rulemaking and in guidance documents that describe in more detail appropriate ways to comply with the regulations. As a result of these efforts, most of our actions on product approval applications are consistent with international standards for data collection and quality of analysis.

      We also are in the process of developing good review practices (GRPs) for drug reviews with the goal of making our drug product review process consistent across all divisions in the Center for Drug Evaluation and Research. A major emphasis of the GRP project is to ensure that the reviews we make available to the public are consistently formatted and clearly written so interested individuals can access important health and safety information.

      We frequently consult with scientific experts on product approval applications and broader issues. We have 31 standing Advisory Committees ⁵, whom we routinely consult on whether the data in particular applications are sufficient to support an approval decision. As noted above, we incorporate our approval decisions into drug approval packages and device summaries of safety and effectiveness that contain our analyses of the submitted data. These packages and summaries do not include confidential commercial, trade secret, and other information exempt from disclosure when we place them on the Internet.

   2. Food Safety Activities

      One of FDA's major areas of responsibility is ensuring the safety of the food we eat. Food safety activities include research, risk assessment, inspections, surveillance, compliance, education, and system coordination activities. We must make sure that the information we provide on food safety is presented in an accurate, clear, complete and unbiased manner. This means that the data on which we base our decisions must be collected in an objective manner using sound scientific principles for data collection.

      We collect information to support our food safety activities through many sources, including research, risk assessment, inspection and surveillance, peer-reviewed literature, and advisory committee opinions. We conduct in-house research on a variety of food safety topics and also fund a substantial amount of extramural research every year. The Center for Food Safety and Applied Nutrition (CFSAN) participates in collaborative research on processing and packaging through the National Center for Food Safety and Technology, which is a consortium of government, industry, and academia, and we coordinate food safety research activities through a cooperative program with the University of Maryland. Topics of both internal and external research interest are determined by CFSAN's 3-year research plan. This plan is developed in conjunction with other federal agencies to prioritize our research to inform our most critical food safety efforts and to avoid duplication of effort.

      We also gather information for certain food safety activities through risk assessment. Risk assessments are a very useful tool for evaluating the benefits of pursuing various rulemaking strategies. To date, we have conducted or been involved in four risk assessments related to food safety. We conducted a joint risk assessment with USDA on Salmonella Enteritidis in shell eggs that was published in 1998. More recently, we published risk assessments on the Public Health Impact of Vibrio parahaemolyticus in Raw Molluscan Shellfish and the Relative Risk to Public Health from Foodborne Listeria monocytogenes Among Selected Ready-to-Eat Foods. We also intend to use the risk assessment model developed for USDA by Harvard University on bovine spongiform encephalopathy to determine the risk reduction outcomes of various rulemaking efforts we are considering.

      We gather data for FDA food safety activities through use of surveys designed for specific purposes and advisory committee opinions. The surveys include the Health and Diet Survey and the Total Diet Study, both of which are used in our development of safety and exposure assessments for various compounds. When information is not available through research or literature, we have several advisory committees that are able to render expert opinions on particular matters. These committees include the National Advisory Committee on Microbiological Criteria for Foods, which considers a variety of food safety issues for FDA and USDA, and FDA's Transmissible Spongiform Encephalopathy Advisory Committee (TSEAC), which specifically considers issues related to TSE diseases. Finally, our food safety activities are informed through the participation of FDA scientists in a variety of professional organizations such as Codex Alimentarius, International Commission on Microbiological Specifications for Foods, Institute of Food Technologists, American Society for Microbiology, the International Association for Food Protection, Society for Toxicology, American Chemical Society, the National Academy of Sciences, International Life Sciences Institute, and editorial review boards of several publications including Journal of Food Protection and Journal of Food Science.

   3. Adverse Events Analysis for Medical Products

      Once products are marketed, we continue to monitor their safety after approval and disseminate information about their risks to health care providers, patients, and consumers. We undertake a number of data collection activities to ensure the objectivity of the information we disseminate on medical products.

      1. Human Drug and Biological Products

         The Adverse Event Reporting System (AERS) is an Oracle-based computerized information system designed to support the Agency's postmarketing safety surveillance program for all approved drug and therapeutic biologic products. The structure of the database complies with the international safety reporting guidance (E2B Guidance on Data Elements for Transmission of Individual Case Safety Reports, January 1998 ⁶), including content and format for electronic submission of the reports from the manufacturers. The ultimate goal of AERS is to help reduce the risks associated with medical product use by providing the best available tools for storing and analyzing safety reports. Information from this system is used to support decisions to disseminate information of product safety. By systematizing the submission of data to the Agency, we have greatly improved the quality (and objectivity) of related decisions and information dissemination efforts.

         A separate system is used to monitor the safety of vaccines after approval. The Vaccine Adverse Events Reporting System (VAERS) is a cooperative program for vaccine safety of the FDA and the Centers for Disease Control and Prevention (CDC). VAERS is a postmarketing safety surveillance program, collecting information about adverse events that occur after the administration of U.S. licensed vaccines. Other systems are in place to monitor the quality of manufacturing of drugs and biological products and blood related products.

      2. Medical Devices

         We use a variety of tools to identify problems and safety issues related to medical devices that are approved and being used by health care practitioners and consumers. Tools include both voluntary and mandatory reporting of adverse events; monitoring of product performance through other data sources, such as registries and various research efforts; and the use of both mandatory and voluntary postmarket studies aimed at examining specific safety issues.

         Adverse events related to either product problems or issues associated with the use of the device are reported by both manufacturers and device users. Although both the manufacturing and user communities have mandatory reporting requirements for device-related problems, this surveillance system is virtually a passive system that depends on the reporter to recognize an event and follow through in reporting. This passive surveillance system is augmented by a more active reporting network composed of hospitals and other health care facilities, where reporting is encouraged and supported through educational activities and feedback. Adverse event reports are immediately triaged to quickly identify problems that require urgent attention; all reports are then reviewed by clinical analysts and others with appropriate expertise to decide if further follow-up is needed.

      3. Animal Drugs

         An Adverse Drug Event (ADE) report for veterinary medicinal products consists of either an undesired side effect or the lack of a desired effect associated with drugs administered to animals. Reports may also involve product defects and potential harm posed to persons administering or using animal drugs. For example, in the year 2000, FDA's Center forVeterinary Medicine reviewed 14,497 ADE reports consisting of: 13,757 undesired side effects and lack of desired effect; and 740 product defects.

         Adverse event reports related to animal drugs are maintained in a separate database, the Center for Veterinary Medicine's adverse drug event reporting system (ADERS). This database is used to identify adverse effects not detected during pre-market testing of FDA-approved animal drugs and to monitor the performance of drugs not approved for use in animals. The ADERS depends upon the detection of an adverse clinical event by veterinarians and animal owners, the attribution of the clinical event to the use of a particular drug ("suspect" drug), and the reporting of the ADE either to the manufacturer of the suspected drug or directly to FDA. Data from these ADE reports are reviewed, coded and entered into the computerized ADERS.

         The ADE for veterinary drugs generates current information on the safety and efficacy of veterinary drugs. These data expand the knowledge base used in animal drug approvals and ultimately contribute to reducing the risks associated with veterinary medical products. Summary information from this system is available to support decisions about disseminating information on product safety.

      4. Foods, Including Dietary Supplements, and Cosmetics

         We use several reporting systems to identify problems associated with foods, including dietary supplements, and cosmetics.

         • The Adverse Reaction Monitoring System (ARMS) collects spontaneous reports from consumers and health professionals regarding alleged adverse effects from food products.
         • The Special Nutritional Adverse Event Monitoring System (SN/AEMS) collects spontaneous reports from consumers and health professionals regarding adverse effects from special nutritional.
         • The Cosmetic Adverse Reaction Monitoring System (CARMS) collects spontaneous reports from consumers and health professionals regarding alleged adverse effects from cosmetic products.
         • CFSAN receives adverse event reports linked to the products it regulates through FDA's MedWatch program.

   4. Research to Support the Regulatory Functions of FDA

      FDA conducts peer-reviewed research that supports the regulatory functions of the Agency. To accomplish this mission, the FDA seeks feedback from its stakeholders and partners, including its center scientists, other government agencies, industry, and academia. In addition, the FDA Science Board (SB) provides guidance on specific complex and technical issues as well as emerging issues within the scientific community, in industry, and academia. The SB conducts ongoing reviews of the research programs themselves and how these programs are keeping pace with technical and scientific evolutions in the fields of regulatory science. Their review includes advice on developing an appropriate research agenda and on upgrading FDA's scientific and research facilities to keep pace with these changes. The SB is composed of non-governmental scientists from industry, academia, and consumer organizations. This board is further supplemented with subject matter experts and scientists representing all FDA centers. Research programs can be via collaborations through partnerships with other scientific organizations. Scientific and monetary collaborations include interagency agreements with other government agencies, Cooperative Research and Development Agreements and technology transfer with industry, and grants or informal agreements with academic institutions.

      Each FDA Center and Office also uses several strategies to ensure the quality (including objectivity) of its extramural research and the accuracy of data collected in association with these studies. For example, study protocols may be developed collaboratively by principal investigators and program officials. A project advisory group (PAG) consisting of experts who oversee extramural research projects in a particular field may be appointed.

      To ensure the data are accurate and timely, FDA monitors research progress at the project level on a recurring basis. FDA quality assurance staff assess experiments that fall within the scope of FDA's Good Laboratory practice guidelines for compliance. Additionally, agency scientists publish research manuscripts, book chapters, and abstracts in recognized, peer-reviewed scientific journals. FDA's research findings are also presented at national and international scientific meetings, many of which are sponsored or co-sponsored by FDA. Many FDA scientists also serve on international scientific advisory boards.

3. Integrity

   We strive to maintain the integrity of the information we collect and use, and protect it against disclosure, alteration, loss, or destruction. We require all of our operating divisions to adhere to a series of Agency guidelines to ensure our data integrity operations. Guidelines include:

   • FDA Staff Manual Guide 3250.17 Data Security\Data Integrity
   • Computer Security Act of 1987
   • Computer Fraud and Abuse Act of 1986
   • Clinger-Cohen Act of 1996
   • Federal Managers Financial Integrity Act
   • Government Information Security Reform Act (GISRA)
   • Office of Management and Budget (OMB) Circular A-130, Appendix III, Management of Federal Information Resources, Revised 2/96
   • HHS Automated Information Systems Security Program Handbook (Release 2.0), 5/94
   • NIST Special Publication 800-14, Generally Accepted Principles and Practices for Securing Information Technology Systems (September 1996)

   All data submitted for inclusion in our systems must be accompanied by information about origin, sensitivity, reliability, and the date of most recent revision. We have systems in place to ensure that data modifications are accomplished in a managed and controlled manner and that all information is protected from unauthorized access, revision, corruption, or falsification.

   Transactions affecting sensitive or valuable information can only be processed if the originating individual or system has been validated as authorized to submit such transactions. Additionally, transactions must be initiated only through source documents or computerized messages in which the originating individual or system is clearly identified. All transactions intended for input into a multi-user production computer system must first be subjected to reasonableness checks, edit checks, and/or validation checks.

   Transactions that fail such checks must either be:

   • Rejected with a notification of the rejection sent to the submitter
   • Corrected and resubmitted or
   • Suspended pending further investigation

   Management has established and maintains preventive and detective security measures that ensure that our information is protected from undetected alteration. All rejected input transactions are placed in a suspense file and listed in exception reports until they are successfully resubmitted for processing. Resubmission and corrections are subject to the same validation procedures that original input transactions receive. Management reviews the reasonableness and accuracy of all changes to internal records. If a client or customer brings record errors to our attention, an investigation of the errors is initiated promptly.

   The Office of Public Affairs provides guidance to the Agency on best communications principles and practices to help assure that public messages are accurate, understandable and consistent with current Agency policies. The quality, objectivity, utility and integrity of information posted on the FDA Internet is assured by the internal clearance process used to clear printed documents.

VI. Agency Administrative Complaint Procedures

1. Format for Submitting Complaints

   As described below, we intend to use existing complaint mechanisms to address complaints from the public concerning our information dissemination activities. You may use any of the mechanisms outlined below to request correction of information disseminated by FDA that you believe does not meet the applicable OMB or HHS Information Quality Guidelines, or this Guidance.

   If you request a correction of any information disseminated by us, we would appreciate it if you clearly designate the request as a request for correction of information under this guidance and the OMB and HHS Guidelines. You should use the following format for your request:

   1. Your name, mailing address, fax number or e-mail address, telephone number, and organizational affiliation, if any, of the requestor,
   2. A detailed description of the specific material that needs to be corrected including where the material is located, i.e., the publication title, date, and publication number, if any, or the website and web page address (Uniform Resource Locator or url),
   3. The specific reasons for believing the information does not meet applicable OMB, HHS, or FDA guidelines and is in error, and supporting documentation,
   4. The specific recommendations for correcting the information,
   5. A description of how the person submitting the complaint is affected by the information error.

2. Procedures for Submitting Complaints

   We have clear procedures in place to address complaints from the public. Requests should be submitted to the Agency in accordance with the procedures described below for dispute resolution (i.e., beginning with the employee or division that disseminated the information, or by contacting the center, the Agency, or an ombudsman). To help us track and monitor complaints, regardless of which procedures you use, we ask that you also send a copy of your request for correction to:

   Office of the Ombudsman
   Food and Drug Administration
   10903 New Hampshire Avenue
   WO Building 1, Room 4208
   Silver Spring, MD 20993

   It will also be helpful to those wishing to submit a request for correction electronically if the Office e-mail address is included: Ombuds@OC.FDA.gov.

   FDA regulations at 21 CFR 10.75 provide a mechanism for any interested person (a person who submits a petition, comment, or objection, or otherwise asks to participate in an informal or formal administrative proceeding or court action) to obtain formal review of any Agency decision or action by raising the matter with the supervisor of the employee who made the decision. These procedures can be used to submit an initial complaint about an FDA information dissemination. If the issue is not resolved at the primary supervisory level, the interested person may request that the matter be reviewed at the next higher supervisory level. This process may continue throughout the Agency's chain of command, through the centers to the Commissioner of the FDA.

   Regulations for dispute resolution during the application review process (21 CFR 312.48; 314.103; and 814.42 (d), 814.46(c), 814.112(b), and 808.25 (e)) specify procedures similar to those outlined above. CDRH also established a Medical Devices Dispute Resolution Panel to hear scientific disputes. Regulations for CDER and CBER also provide that a sponsor may request that we seek the advice of outside experts. In addition, we may refer major issues to an appropriate advisory committee for its recommendations (§§ 312.48(c)(3) and 314.103(c)(3)).

   Several guidances explaining the dispute resolution process also are available:

   • Formal Dispute Resolution: Appeals Above the Division Level (for drug and biological products, February 2000),
   • Resolving Scientific Disputes Concerning the Regulation of Medical Devices, a Guide to Use of the Medical Devices Dispute Resolution Panel (July 2001)
   • Medical Device Appeals and Complaints: Guidance on Dispute Resolution (February 1998)
   • A Suggested Approach to Resolving Least Burdensome Issues (September 2000)

   Finally, 21 CFR 5.200 provides for the establishment of an Agency ombudsman. We have established an Ombudsman Office within the Office of the Commissioner, and each center has identified or is identifying an ombudsman. Information about when and how to contact an Agency or center ombudsman can be found on our Internet site ⁷. We encourage interested parties who may be reluctant to contact a program person in a specific program, division, office or center to feel free to contact a center or Agency ombudsman.

   Existing public comment procedures for rule-makings and other formal agency actions already provide well established procedural safeguards that allow affected persons to raise information quality issues on a timely basis. Accordingly, FDA will use these existing procedures to respond to information quality complaints that arise in this process.

   In cases where the agency disseminates a study, analysis, or other information prior to the final agency action or information product, requests for correction will be considered prior to the final agency action or information product in those cases where in the agency's judgment issuing an earlier response would not unduly delay issuance of the agency action or information product and the complainant has shown a reasonable likelihood of suffering actual harm from the agency's dissemination if the agency does not resolve the complaint prior to the final agency action or information product.

   You should be aware that you bear the "burden of proof" with respect to your request for correction.

3. FDA's Response to Complaints

   Based on a review of the information you provide in your complaint, we will determine whether a correction is warranted and, if so, what action, if any, we will take. We will respond to you in a manner appropriate to the nature and extent of your complaint (for example, by letter, e-mail, fax, press release, mass mailing) and will indicate whether, and if so how, we intend to correct the information. We will respond in accordance with the time frame specified in the complaint mechanism you use. Where a procedure does not specify a time frame for a response, we will respond within 60 days, in accordance with the OMB and HHS Guidelines. We will respond by either issuing a decision or by informing you that more time is required, explaining why, and providing you with an estimated decision date. In deciding how to respond, we will consider all of our legal obligations and policies.

4. Requests for Reconsideration

   If you do not agree with FDA's decision about your complaint (including any corrective action), you may send a request for reconsideration within 30 days of receipt of our decision. You may use any of the Procedures for Submitting Complaints described above. A request for reconsideration should state the reasons why you believe the response is inadequate, should be designated as an "Information Quality Appeal," and should include a copy of your original request and the agency's decision. The agency will respond to all requests for appeals within the time frame specified in the procedure you use. Where a procedure does not specify a time frame for a response to your appeal, we will respond in a timely manner, in accordance with the OMB and HHS Guidelines.

VII. Influential Scientific, Financial, and Statistical Information

As illustrated by the number and types of information we disseminate and the variety of methods we use to disseminate them, it is clear that we strive for a high degree of transparency with regard to all of our information dissemination activities. The OMB Guidelines, however, apply special quality standards to the dissemination of information that is considered influential. Such information must meet high standards of transparency of the data and methods used to facilitate the reproducibility of such information by third parties. As defined below, influential scientific, financial and statistical information that FDA disseminates will meet the high standards in the OMB Guidelines for such information.

1. Definition of the Term Influential

   The term influential information, when used in the OMB Guideline in the phrase "influential scientific, financial, or statistical information," applies when the agency can "reasonably determine that dissemination of the information will have or does have a clear and substantial impact on important public policies or important private sector decisions" (67 FR 8452; February 22, 2002). However, because each agency is different, and there are vast differences in the types of information they disseminate, each agency has been asked to elaborate on the definition of influential in the context of their missions and duties, "with due consideration of the nature of the information they disseminate." As stated in the OMB Guideline (V.9), "[e]ach agency is authorized to define 'influential' in ways appropriate for it given the nature and multiplicity of issues for which the agency is responsible."

   For purposes of this guidance, influential information is defined as disseminated information that results from or is used in support of agency actions that are expected to have an annual effect on the economy of $100 million or more or will adversely affect in a material way the economy, a sector of the economy, productivity, competition, jobs, the environment, public health or safety, or State, local or tribal governments or communities. It should be noted that the definition applies to "information" itself, not to decisions that the information may support. Even if a decision or action by FDA is itself very important, a particular piece of information supporting it may or may not be "influential."

   Two examples of what FDA considers to be influential information follow.

   • Quality Mammography Standards

     On October 28, 1997, we issued a final rule (62 FR 55852) amending our regulations governing mammography to provide increased assurance of adequate and consistent evaluation of mammography facilities on a nationwide level and compliance of the facilities with quality standards. Costs of the regulation include replacing below standard mammography units and film processors, providing written results of tests to patients, providing telephone results of tests to referring physicians, and conducting required weekly image quality tests. Because this rulemaking was expected to have an annual effect on the economy of more than $100 million, the information FDA disseminated in support of this rulemaking would be considered influential information.

   • Hazard Analysis and Critical Control Point (HACCP); Procedures for the Safe and Sanitary Processing and Importing of Juice

     On January 19, 2001, we adopted a final rule (66 FR 6138) to ensure the safe and sanitary processing of fruit and vegetable juices. The regulations mandate the application of HACCP principles to the processing of these foods. HACCP is a preventive system of hazards control. FDA adopted this rule in response to a number of food hazards associated with juice products and because preventive control measures are the most effective and efficient way to ensure that these products are safe. The final regulation involves costs to the manufacturers and processors of juice products for implementing procedures consistent with the regulation that exceeded $100 million. Therefore, the information disseminated in connection with this rulemaking would be considered influential information.

2. Transparency

   If information that meets the criteria for influential information is disseminated, the OMB Guidelines provide that it must meet certain higher standards of transparency and methods to facilitate the reproducibility of information by qualified third parties. When we disseminate information, but particularly in those cases involving influential information, we strive to ensure that the information is accurate and unbiased, as well as substantially reproducible and replicable. This goal is accomplished by using reliable data sources and sound analytical techniques, and by employing a high degree of transparency about the data, methods, measures, assumptions and limitations used to develop the information to facilitate reproducibility by third parties. Our goal is to provide a clear explanation of the assumptions and data upon which we base our conclusions, the criteria used to determine the suitability of the data for use, the methods used in our analysis, and the conclusions we have drawn.

   Because of legal obligations to maintain the confidentiality of data supplied by third parties, there may be instances when original or supporting data may not be available to the public. In such cases, we will disclose the specific data sources used and the specific quantitative methods and assumptions employed. We will also conduct especially rigorous robustness checks of any models used in the analysis and the analytic results so that there will be a high degree of confidence in the results.

   Biases, if any, will be revealed. All assumptions used in the analysis, the scientific rationale, and data used to estimate the impact of the various factors influencing the analysis should be clearly stated. This ensures that biases will be eliminated or minimized and that any introduced biases will be clearly identified.

   Clarity includes ensuring the information disseminated is clear and understandable. When detailed technical information is needed to provide sufficient information so that a qualified third party could reproduce the analysis, the resulting document may be lengthy and difficult the public to understand. One approach that can provide additional transparency in such cases is to develop an interpretative summary document as a companion to the technical analysis. The summary document can provide a non-technical explanation of the data, process, results, and conclusions in a manner that the public can understand. As discussed under "Objectivity," we have a strong commitment to writing all our new documents in plain English. As we revise and update existing documents, we will ensure that they are written in plain English. Our goal is to make our written communications more understandable.

   A participatory process should be used. The process for generating information defined as influential should be transparent. One approach is to invite public comment on the information to be disseminated and encourage stakeholders to submit scientific data and information that can be used in preparing the information. As appropriate, we will solicit advice and opinions of advisory committees as well as peer review from experts within and outside of the agency. To the extent practicable under confidentiality laws, we will strive to make supporting data and analyses available to the public for technical review and comment. This can be accomplished by posting the information on our web pages and providing printed copies as requested.

3. Risk Assessment

   Some of the influential information that we disseminate is based on an analysis of the risks to the public of certain actions or exposures to hazardous substances. For purposes of this guidance, we are defining risk as the likelihood that injury or damage is or can be caused by a substance, technology, or activity. We use risk analysis (the integration of risk assessment with risk management and risk communication) as a tool to enhance the scientific basis for all of our regulatory decisions.

   The OMB Guidelines provide that in addition to the ordinary standards for utility, objectivity, and integrity that apply to dissemination of information, special considerations must be taken into account in certain risk assessments, i.e., those that provide the basis for the dissemination of influential information. The Guidelines state that "With regard to analysis of risks to human health, safety, and the environment maintained or disseminated by the agencies, agencies shall either adopt or adapt the quality principles applied by Congress to risk information used and disseminated pursuant to the Safe Drinking Water Act Amendments of 1996 (SDWA) (42 U.S.C. 300g-1(b)(3)(A) and (B)).

   The SDWA risk assessment principles are as follows:

   1. To the degree that the agency action is based on science, the agency shall use
      1. the best available, peer-reviewed science and supporting studies conducted in accordance with sound and objective scientific practices
      2. data collected by accepted methods (if reliability of the method and the nature of the decision justify use of the data)
   2. In the dissemination of public information about risks, the agency shall ensure that the presentation of information about risk effects is comprehensive, informative, and understandable.
   3. In a document made available to the public in support of a regulation, the agency shall specify, to the extent practicable
      1. Each population addressed by any estimate of applicable risk effects
      2. The expected risk or central estimate of risk for the specific populations affected
      3. Each appropriate upper-bound or lower-bound estimate of risk
      4. Each significant uncertainty identified in the process of the assessment of risk effects and the studies that would assist in resolving the uncertainty and
      5. Peer-reviewed studies known to the agency that support, are directly relevant to, or fail to support any estimate of risk effects and the methodology used to reconcile the inconsistencies in the scientific data

   Many of our actions are based on scientific experts' judgments using available data, are essentially qualitative, and are generally carried out for non-cancer-causing hazards. Such assessments provide useful answers in most instances that are sufficient for regulatory purposes, and much more elaborate, quantitative estimates extrapolating beyond the data are unnecessary. For example, we may issue regulations on submission requirements for product approval applications, electronic submission of product labeling, or periodic reporting by manufacturers of adverse events from drugs; devices; and biologics, including blood, vaccines, and tissues. Regulations like these do not always lend themselves to the types of quantitative risk assessments contemplated by the Safe Drinking Water Act principles.

   Other actions are based on research and supporting data that are generated outside FDA. For example, most product approval actions are based on scientific studies conducted by sponsors seeking marketing approval in accordance with our regulations and guidance documents. Our regulations and guidance documents describe sound scientific practices for conducting human and animal studies of medical products and analyzing the resulting data. Most information in these studies is considered confidential commercial information and is closely held by the sponsors. As a result, formal peer-review of the data is rare. However, for certain drug approval applications, the safety and/or effectiveness information is presented to scientific advisory committees for recommendations. Evaluations of food safety and nutritional data are also presented to scientific advisory committees.

   As a result, we have adapted the general principles for risk assessments from the SDWA to fit these situations. The principles we intend to apply to risk assessments involving the dissemination of influential information affecting product approval actions or regulations that do not lend themselves to quantitative risk assessment are as follows:

   1. The Agency will use
      1. the best available science and supporting studies conducted in accordance with sound and objective scientific practices, including peer reviewed science and supporting studies when available
      2. data collected by accepted methods (if reliability of the method and the nature of the decision justify use of the data)
   2. In the dissemination of public information about risks, the Agency will ensure that the presentation of information about risk effects is comprehensive, informative, and understandable.

   In situations requiring a quantitative risk assessment, we generally follow basic risk assessment principles in the NAS paradigm of 1983. Our needs for quantitative risk assessments range over a wide variety of hazards including physical hazards encountered during use of a medical device, food chemical residues, and antimicrobial resistance genes in bacteria. Thus, we also ascribe to the statement from NAS when it revisited the risk assessment process in 1994 (Science and Judgment in Risk Assessment, NAS 1994): "Risk assessment is not a single process, but a systematic approach to organizing and analyzing scientific knowledge and information. "In each of the areas we regulate, we apply risk assessment practices to the specific task that are widely accepted among relevant domestic and international public health agencies.

   For quantitative risk assessments in support of the dissemination of influential information, FDA intends to apply the following principles:

   1. The agency will use-
      1. the best available science and supporting studies conducted in accordance with sound and objective scientific practices, including peer reviewed science and supporting studies when available;
      2. data collected by accepted methods (if reliability of the method and the nature of the decision justifies use of the data)
   2. In the dissemination of public information about health risks, the agency shall ensure that the presentation of information is comprehensive, informative, and understandable, within the context of its intended purpose.
   3. In a risk assessment document made available to the public, the agency shall specify, to the extent practicable-
      1. Each population addressed by any estimate of applicable effects;
      2. The expected or central estimate of risk for the specific populations affected;
      3. Each appropriate upper-bound and/or lower-bound risk estimate and the methodology used to reconcile the inconsistencies in the scientific data;
      4. Data gaps and other significant uncertainties identified in the process of the risk assessment and the studies that would assist in characterizing the uncertainties; and
      5. Additional studies not used to produce the risk estimate that support or fail to support the findings of the assessment, and the rationale of why they were not used.

VIII. Special Considerations for Agency Dissemination

Under certain circumstances, we may need to disseminate information without fully applying the principles for ensuring the quality, objectivity, utility and integrity of the information outlined above. Even in these cases, however, FDA intends to use its internal review process to evaluate the data received and the information it plans to disseminate to ensure to the degree practicable the accuracy, objectivity, and transparency of the relevant information, and will entertain requests for correction after the exigent circumstances have passed. The specific situations where this may occur are as follows:

• Public Health Emergencies: In the case of a public health emergency, there may not be time for the Agency to submit relevant information to all levels of review or review by an Advisory Committee prior to dissemination of the information.
• Statutory or Other Legal Requirement: If a statutory requirement, Executive Order, or court order requires immediate implementation of a policy, we may have insufficient time to apply the OMB requirements prior to disseminating information relevant to that policy.
• Other Circumstances: There may be unforeseen circumstances in carrying out our mission that could prevent the Agency from applying all of the OMB guidelines when disseminating information to the public.

As mentioned above, in all such special circumstances, the Agency will be particularly careful to use its internal review processes to the extent practicable when considering the dissemination of relevant information to the public.

IX. References

Clinger-Cohen Act of 1996.
Code of Federal Regulations (CFR), Title 21, Parts 5, 10, 50, 56, 58, 312, and 314.
Computer Fraud and Abuse Act of 1986.
Computer Security Act of 1987.
Consolidated Appropriations Act, 2001 (Public Law 106-554), Appendix C -- H.R. 5658.
HHS Automated Information Systems Security Program Handbook (Release 2.0), May 1994.
FDA guidance, Formal Dispute Resolution: Appeals Above the Division Level, February 2000.
FDA guidance, Medical Device Appeals and Complaints: Guidance on Dispute Resolution, February 1998.
FDA guidance, Resolving Scientific Disputes Concerning the Regulation of Medical Devices, a Guide to Use of the Medical Devices Dispute Resolution Panel, July 2001.
FDA Internet world Wide Web Site Guidelines, rev. March 23, 2001.
FDA Staff Manual Guide 3250.17,Data Security\Data Integrity.
Federal Food, Drug, and Cosmetic Act (21 U.S.C. 301 et seq.).
Federal Managers Financial Integrity Act.
Food and Drug Administration Modernization Act of 1995 (Public Law 105-115).
Freedom of Information Act and Electronic Freedom of Information Act Amendments (5 U.S.C. 552).
Government Information Security Reform Act (GISRA).
ICH guidance, E2B Guidance on Data Elements for Transmission of Individual Case Safety Reports, January 1998.
NIST Special Publication 800-14, Generally Accepted Principles and Practices for Securing Information Technology Systems, September 1996.
Office of Management and Budget (OMB) Circular A-130, Appendix III, Management of Federal Information Resources, rev. February 1996.
Public Health Service Act (42 U.S.C. 201 et seq.).
Regulatory Flexibility Act (5 U.S.C. 601-612).
Safe Drinking Water Act Amendments (Public Law 104-182).
Treasury and General Government Appropriations Act, 2001.

Endnotes

¹ These guidelines published in the Federal Register on January 3, 2002 (67 FR 369) and were republished with corrections on February 22, 2002 (57 FR 8452).

² All product reviews undergo extensive review through a hierarchical process (see Section V).

³ Good laboratory practices (GLPs) for nonclinical laboratory studies are discussed in 21 CFR 58.

⁴ Regulations at 21 CFR 312, guidances developed as part of the Agency's international harmonization efforts (for example, E6 and E8), and guidances developed by FDA that address clinical development of drugs to treat specific indications provide requirements and recommendations on good clinical practice (GCP). In addition 21 CFR parts 50 and 56 address issues related to informed consent and investigational review boards (IRBs), respectively.

⁵ FDA also administers an HHS Advisory Committee that has 18 panels.

⁶ This guidance is being updated and should be available soon.

⁷ Information on the FDA Ombudsman Program can be found at www.fda.gov/oc/ombudsman/homepage.htm.

1. Agency Mission

   Tens of millions of Americans receive quality, affordable health care and other services through the Health Resources and Services Administration (HRSA), an agency within the U.S. Department of Health and Human Services (HHS). HRSA programs provide health care to people who are geographically isolated and economically or medically vulnerable. This includes programs that deliver health services to people with HIV, pregnant women, mothers and their families, those with low incomes, residents of rural areas, American Indians and Alaska Natives, and those otherwise unable to access high-quality health care. HRSA programs also support health infrastructure, including through training of health professionals and distributing them to areas where they are needed most, providing financial support to health care providers, and advancing telehealth. In addition, HRSA oversees programs for providing discounts on prescription drugs to safety net providers, facilitating organ, bone marrow, and cord blood transplantation, compensating individuals injured by vaccination, and maintaining data on health care malpractice payments.

   More information on HRSA’s Bureaus and Offices and on HRSA’s programs and activities is available on the HRSA website.

2. Scope and Applicability of Guidelines for Agency/Office

   HRSA works to ensure that the information the agency disseminates meets the standards of quality set forth in the Information Quality Act as well as in Office of Management and Budget (OMB), HHS, and HRSA guidelines. It is HRSA's policy to ensure and maximize the quality, objectivity, utility, and integrity of information that it disseminates to the public. We strive to provide information that is accurate, reliable, clear, complete, unbiased, and useful. We are committed to integrating the principle of information quality into every phase of information development, including creation, collection, maintenance, and dissemination.

   The guidelines apply to substantive information that is disseminated by Federal agencies subject to the Paper Reduction Act (PRA), excluding information that is someone's opinion rather than fact or the agency's views. More specifically, it applies to agency initiated or sponsored information. The HRSA information to which these guidelines would apply are aggregate statistical types of information that are abstracted from grantee applications, policy and program information (i.e., data and state profile information, community information, official reports), as well as other electronic documents, newsletters, and brochures that are provided to the public. For example, this includes the Title V Information System website, the current findings from the National Sample Survey of Registered Nurses, and the Area Health Resources Files.

   The pre-dissemination review described in the guidelines only applies to information first disseminated on or after October 1, 2002. The administrative mechanism for correction applies to information that the agency disseminates on or after October 1, 2002, regardless of when the agency first disseminated the information.

   The guidelines would not include data provided directly to HRSA by grantees, such as individual grantee application forms, reporting requirements, performance plans, etc., but would apply to information that is disseminated from such collections of information. An example of another type of information collection to which these guidelines would not apply is the information that is collected for the National Practitioner Data Bank (NPDB), since this information is not disseminated to the public but is restricted in access to those entities and persons who, under specified conditions may request information from the NPDB. Other information that is not covered includes papers, books, journal articles, and other documents that have the disclaimer that the information contained therein does not represent the agency's views. Information that is not covered also includes press releases, correspondence with individual people, archival records, or opinions. In addition, the guidelines do not apply to press releases that support the announcement or give public notice of information that HRSA has disseminated elsewhere.

3. Types of Information Disseminated by the Agency to the Public HRSA disseminates a range of information to the public via the HRSA website, the HRSA Data Warehouse, and HRSA eNews. The HRSA website also provides information, data, and reports on community health resources, health professions training, HIV/AIDS, maternal and child health, oral health, primary health care, and rural health. Within the HRSA website there is a wealth of public information.
   1. Examples of program, policy, and administrative data:
      1. The Title V Information System (TVIS): The TVIS provides information on the status of maternal and child health in the United States. Data are compiled from annual Title V Block Grant applications and reports submitted by all U.S. States, Territories, and Jurisdictions. Information is provided on key measures of maternal and child health, budgets and expenditures, program data, performance measures, and summary data. Users may download prepared tables, graphs, fact sheets, and reports on maternal and child health.
      2. HRSA's Bureau of Health Workforce Area Health Resources Files (AHRF): The AHRF is a county level data set with information on health professions, health facilities, utilization, expenditures, population characteristics, and geographic environment. The data set includes county level estimates for the entire U.S. of the number of physicians, including specialties and limited demographic information, dentists, dental hygienists, optometrists, pharmacists, podiatrists, nurses, physician assistants, and other health professions. Information is available on hospitals, nursing homes, HMOs, inpatient days and other utilization measures, hospital and Medicare expenditures, and various selected variables containing information on the resident population and environmental characteristics. The AHRF utilizes secondary data obtained from Census, the American Medical Association, National Center for Health Statistics, and others and compiles a wide array of data sources. As such, the limitations of these data sources all apply, as do any cross-source comparability issues.
      3. Ryan White HIV/AIDS Program (RWHAP) Services Report (RSR): To provide context in the next section, this report is an example of the HIV/AIDS Bureau (HAB) data collection report. The RSR is HABs primary source of annual client-level data. More than 2,000 grant recipients and subrecipients report these data each year. The data allows HRSA HAB and its stakeholders to assess who is served, including client characteristics, service use, and HIV-related outcomes. De-identified client-level data have been collected since 2010 with publication since 2015. Each year, recipients and subrecipients funded by the RWHAP to core medical or support services during the reporting period must submit data in a standardized format.
   2. Example of statistical data: The National Sample Survey of Registered Nurses (NSSRN): The NSSRN is the nation's most extensive and comprehensive source of statistics on all those with current licenses to practice in the United States. It provides information on the number of registered nurses, their educational background and specialty areas; their employment settings, position levels, and salaries; their geographic distribution; their personal characteristics including gender, racial/ethnic background, age, family status, and satisfaction with their job. The substantial database resulting from this study may be used for many different types of analyses concerning a variety of subjects. The report presents an overview of the personal, professional, and employment characteristics of the millions of registered nurses in the country. A summary of the findings from the study and some comparisons to the findings of prior studies in this series are presented in the report. Appendix A contains a series of tables summarizing the data. A review of the survey methodology and the statistical techniques used in sample selection, response weighting, and identification of sampling errors are found in Appendix B. The survey instrument is included in Appendix C of the report.

4. Types of Dissemination Methods

   HRSA disseminates information using a variety of mechanisms, including the HRSA website, the HRSA Data Warehouse, and HRSA eNews. In addition, for each of the earlier examples given above of information disseminated by the Agency to the public, the following dissemination methods are used:

   1. The Title V Information System is available on the HRSA Maternal and Child Health Bureau website. Program materials can be downloaded to obtain prepared tables and graphs, fact sheets, annual reports, and more. The website also includes specific contact information for questions regarding the content of the site and questions, problems or comments regarding the functionality of the site.
   2. The Area Health Resources Files are available in electronic format only in the HRSA Data Warehouse. Contact information is provided on the HRSA website and includes an address, phone number, fax number, email, and website specific to AHRF.
   3. Findings from the National Sample Survey of Registered Nurses are provided for viewing and download on the HRSA website. Contact information for the HRSA Bureau of Health Workforce is also provided.

5. Agency Quality Assurance Policies, Standards and Processes for Ensuring the Quality of Information Disseminated to the Public

   HRSA reviews the quality (including the objectivity, utility, and integrity) of information before it is disseminated and treats information quality as integral to every step of the development of information, including its creation, collection, maintenance and dissemination.

   1. Program information:

      Program information provided by grantees in the form of application data, reporting requirements, performance reports and progress reports, undergoes a series of reviews prior to any dissemination to the public. Grantees conduct internal reviews to provide data that are accurate, consistent, and complete. The HRSA Bureau or Office conducts a standard internal review to ensure data quality, completeness, and reliability.

   2. Reports, publications, and other products:

      HRSA publications, digital products, and exhibits are required to be consistent with government-wide and HHS public affairs policies. HRSA has published guidelines in the HHS Public Affairs Policies and Standards, which closely follow the regulations described in the Government Printing and Binding Regulations, to assist program staff in meeting requirements for and obtaining appropriate review and clearance of information products.

   3. Statistical data disseminated to the public:

      At HRSA, the quality assurance process begins at the inception of the information development process.

      For information disseminated from sample surveys and other research related or evaluative activities, the widely accepted standards of technical and scientific review are utilized to ensure data quality. These standards include, where appropriate, peer review, internal expert review, Institutional Review Board (IRB) review and OMB review.

      For each of the examples provided earlier, the following quality assurance procedures are used.

      1. HRSA’s RWHAP collects program data from recipients and providers on demographics and services through an electronic reporting portal. HRSA’s HIV/AIDS Bureau (HAB) conducts rigorous data checks and validations to ensure the production of high-quality data and data products for the Ryan White HIV/AIDS Program. At a high level, the validation process consists of two phases: system-level and manual-level review. System-level validation occurs as the initial step within HRSA’s Electronic Handbooks during the data submission period, ensuring that major and routine errors or inconsistencies are identified and addressed at the point of submission (e.g., Ryan White HIV/AIDS Program Services Report data). Following the completion of data submission, HAB performs comprehensive manual validation to identify and resolve any atypical errors or issues requiring closer reviews. This phase involves statisticians and data analysts within HAB, ensuring that the final data outputs meet the highest standards of quality and are suitable for a range of analytical and reporting purposes.
      2. The Maternal and Child Health Bureau Title V annual report uses the Title V Information System (TVIS) that was developed in collaboration with the states to improve accuracy and completeness of the report. The TVIS provides automatic calculations of ratios, rates, and percentages, and carries data over from year to year, and assures that data used in multiple tables is entered only once. The TVIS has a mechanism that warns states when conflicting or unacceptable figures appear as a quality check on the final data. Knowledgeable MCHB program staff monitor and review all information submitted in the grant annual report for completeness, accuracy, and reliability.
      3. The Area Health Resources Files provide county level data from a variety of data sources. As a result, the data have several limitations and users of the file are provided with a disclaimer as to the applicability and detail of the data. The AHRF utilizes secondary data only, and the years and detail for which all county level data are available are considerably limited. Each data source conducts its own review prior to the release of information. Knowledgeable staff within the originating office and across offices conduct standard reviews for completeness and reliability of the data.
      4. The National Sample Survey of Registered Nurses received extensive review during each phase of operation: survey design, sample selection, fielding of the survey, data editing, and analysis. Clearance was required of all proposed sampling methods, survey procedures, the data collection instrument, and analytic plans by HRSA, HHS, and the OMB. This clearance required a series of expert reviews. A scientific review, an internal review, and an IRB review were conducted to ensure that survey procedures, data collection, editing, and analyses would adhere to the highest standards.

      As in all research activities involving human subjects, it is HRSA's policy that adequate protection of participants be ensured in accordance with the provisions of 45 CFR Part 46. Project plans and procedures must be submitted to the appropriate IRB for approval, and the program must execute all necessary Assurances of Compliance with the HRSA Office of Planning, Analysis, and Evaluation. The NSSRN obtained IRB review and approval for all survey methods and procedures.

      For the NSSRN, widely accepted standards as established by the statistical community are utilized to assure data survey quality. The information disseminated in the report provides extensive detail on the analytic issues (sampling and non-sampling errors, standard error calculation, variance estimation, etc.). As with any sample survey, the results are subject to sampling error. The report provides details on measures of variability, standard errors, information on the computation of the sampling variance and the design effect.

      Further, HRSA strives to demonstrate in its PRA clearance packages that each draft information collection will result in information that will be collected, maintained, and used in a way that is consistent with the Information Quality act as well as OMB, HHS and HRSA information quality guidelines.

   4. Scientific Activities, Including Publication of Research Findings in Scientific Journals:

      HRSA abides to the tenets outlined in Executive Order 14303, Restoring Gold Standard Science, which aims to ensure federal agencies conduct scientific activities in a manner that is:

      • Reproducible
      • Transparent
      • Communicative of error and uncertainty
      • Collaborative and interdisciplinary
      • Skeptical of its findings and assumptions
      • Structured for falsifiability of hypotheses
      • Subject to unbiased peer review
      • Accepting of negative results as positive outcomes
      • Without conflicts of interest

      These nine tenets of Gold Standard Science safeguard the credibility of federal research and scientific endeavors. HRSA and other HHS agencies are dedicated to upholding the tenets of Gold Standard Science in all aspects of its scientific activities, programs, and policies. You can read more details about the tenets in guidance from the White House Office of Science and Technology Policy (OSTP).

6. Agency Administrative Complaint Procedures

   1. Responsibility of the Complainant

      To seek a correction of information disseminated by the agency, individuals should follow the procedures described below.

      1. A complaint or request for review and correction of information shall be in written hard copy or electronic form;
      2. it shall be sent to the agency by mail or email; and
      3. it shall state that an information quality request for correction is being submitted.

      The complaint shall contain

      1. a detailed description of the specific material that needs to be corrected including, as appropriate, where the material is located – such as the publication title, date, publication number (if any), the website and web page address (URL) – or the speech title, presenter, date and place of delivery;
      2. the specific reasons for believing that the information does not comply with the Information Quality Act, or OMB, HHS or HRSA guidelines and is in error – with supporting documentation, as appropriate;
      3. the specific recommendations for correcting the information;
      4. a description of how the person submitting the complaint is affected by the information error; and
      5. the name, mailing address, telephone number, e-mail address, and organizational affiliation, if any, of the individual making the complaint.

      Complainants should be aware that they bear the ‘burden of proof’ with respect to the necessity for correction as well as with respect to the type of correction they seek. Complaints by mail should be directed to:

      Health Resources and Services Administration (HRSA)
      HRSA Chief Data Officer 
      Office of Planning, Analysis, and Evaluation (OPAE) 
      5600 Fishers Lane
      Rockville, MD 20857

      Email complaints should be sent to infoquality@hrsa.gov.

   2. Responsibility of the Agency

      Based on a review of the information provided, the agency will determine whether a correction is warranted and, if so, what action to take. The agency will respond to the requestor by letter or email. The agency's response will explain the findings of the review and the actions that the agency will take, if any. The response will consider the nature and timeliness of the information involved and such factors as the significance of the correction on the use of the information and the magnitude of the correction. The response will describe how the complainant may request reconsideration. The agency will respond to all requests for correction within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason that a longer period of time is required to reach a resolution, along with an estimated decision date.

   3. Appeals

      If the individual submitting the complaint does not agree with the agency's decision (including the corrective action, if any), the complainant may send a written hard copy or electronic (email) request for reconsideration within 30 days of receipt of the agency's decision. The appeal shall state the reasons why the agency response is insufficient or inadequate. Complainants shall attach a copy of their original request and the agency response to it, clearly mark the appeal with the words “Information Quality Appeal,” and send the appeal to infoquality@hrsa.gov or the agency appeals address:

      Health Resources and Services Administration (HRSA)
      HRSA Chief Data Officer
      Office of Planning, Analysis, and Evaluation (OPAE)
      5600 Fishers Lane
      Rockville, MD 20857

      The agency official who resolved the original complaint will not have responsibility for the appeal. The agency will respond to all requests for appeals within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.

   4. Comments and Requests for Information:

      HRSA maintains a webpage to answer questions and address comments on subject-specific information. This office has served as the point of entry for queries regarding HRSA information, requests for publications, comments and corrections to information disseminated to the public. To date this office has not received any complaints about information disseminated by HRSA. Calls have been made on occasion, however, by States and awardees to correct and/or update information that is maintained on the HRSA website. These infrequent calls are directed toward the appropriate HRSA Bureau or Office for review, verification, and revision (as appropriate). The States/awardees verify the correction/modification of their data with Bureau or Office staff and after confirmation with program staff the revisions are implemented. In addition, the HRSA website contains a detailed list of HRSA phone numbers by Office and Bureau for comment or for additional information. Comments and questions can be communicated through the options provided on the HRSA website (https://www.hrsa.gov/about/contact).

7. Influential Scientific, Financial and Statistical Information

   From time-to-time, HRSA disseminates information that would generally be regarded as influential. In those circumstances, we use the highest standards of reproducibility and transparency for such information to be disseminated to the public.

8. Other Agency Specific Policies and Procedures

   HRSA, through its Bureaus and Offices, administers a variety of service delivery and demonstration programs. HRSA Bureaus and Offices also administer some specific research programs and a variety of epidemiological and service utilization studies, as well as evaluations which might be considered research. Participants must be protected from potential risks that may be associated with any such research projects, in accordance with the provisions of 45 CFR Part 46. Project plans and procedures must be submitted to the appropriate IRB for approval, and the program must execute all necessary Assurances of Compliance with the HRSA Office of Planning, Analysis, and Evaluation to protect human subjects in research programs conducted or supported by HRSA. This includes guidance on the exemption for certain research activities or public benefit or service programs, requiring evaluation of such claims by program staff and approval by a HRSA Human Subjects Protection Committee.

   If members of the public check the HRSA resources for public information but they do not find adequate information, they may file a Freedom of Information Act (FOIA) request. The HRSA FOIA Home Page has the location and contact information for making such requests, including the HHS FOIA Library and FOIA Reports.

I. Agency Mission

The Indian Health Service (IHS) is responsible for providing federal health services to American Indians and Alaska Natives. The IHS provides a comprehensive health services delivery system for American Indians and Alaska Natives with opportunity for maximum tribal involvement in developing and managing programs to meet their health needs. The goal of the IHS is to raise the health status of American Indian and Alaska Native people to the highest possible level. IHS services are provided directly and also through tribally contracted and operated health programs.

II. Scope and Applicability of Guidelines for IHS

IHS will ensure that disseminated information meets the standards of quality set forth in the OMB, HHS and IHS guidelines. It is IHS’s goal to ensure and maximize the quality, objectivity, utility, and integrity of information that it disseminates to the public. We strive to provide information that is accurate, reliable, clear, complete, unbiased, and useful. We are committed to integrating the principle of information quality into every phase of information development, including creation, collection, maintenance, and dissemination.

The guidelines apply only to the dissemination of substantive information that the IHS initiates or sponsors. The IHS information to which these guidelines would apply are programmatic statistics and IHS program and policy information.

The pre-dissemination review described in the guidelines only applies to information disseminated on or after October 1, 2002. The administrative mechanism for correction applies to information that the agency disseminates on or after October 1, 2002, regardless of when the agency first disseminated the information.

1. Information that is subject to the Information Quality Guidelines includes:
   • Statistical clinical information
   • Actuarial information
   • Epidemiological information
   • Studies and summaries prepared for public dissemination to inform the public about the impact of IHS programs
   • Studies and summaries prepared for use in formulating broad program policy
2. Information that is not subject to the Information Quality Guidelines includes:
   • Personal information received as a result of agency adjudicative decisions
   • Information pertaining to basic agency operations
   • Program publications, such as pamphlets and notices, that describe Area and regional programs
   • Clinical program information
   • Information on how to apply for third party program benefits
   • Procedural and policy manuals
   • Management information that is produced primarily for internal use
   • Press releases that support the announcement or give public notice of information that IHS has disseminated elsewhere.

III. Types of Information Disseminated by the IHS to the Public

IHS disseminates statistical reports and IHS policy and programmatic information to the public via the IHS web site and through IHS publications. The IHS website also contains links to other useful sites relating to health information and IHS related activities. The Indian Health Service (IHS) provides information to the public about the current IHS programs, clinical epidemiological data on American Indian and Alaska Natives(AI/AN), the projected scope and impact of IHS programs of AI/AN people in the future, and the effect of proposed changes to the IHS and tribal health care systems. Information products describe the impact of IHS programs on American Indian and Alaska Natives communities and upon its beneficiary populations. IHS provides detailed demographic and economic information on health benefits for AI/AN people. The IHS also provides information on research among AI/AN communities. IHS information products are used by government planners and policymakers as well as many actuaries, economists and other social scientists, the media and the public to analyze IHS programs and their impact on the health of the Indian Nation.

IV. Types of Dissemination Methods

IHS disseminates information in print and electronic form.

V. Agency Quality Assurance Policies, Standards and Processes for Ensuring the Quality of Information Disseminated to the Public

The quality assurance process begins at the inception of the information development process.  Further, IHS reviews the quality (including the objectivity, utility, and integrity) of information before its is disseminated and treats information quality as integral to every step of the development of information, including its creation, collection, maintenance and dissemination.  In addition, IHS demonstrates in its Paperwork Reduction Act clearance submission that information will be collected, maintained, and used in a way that is consistent with 0MB, HHS and IHS information quality guidelines.

1. Administrative and program data:

   Information is provided by tribal governments in the form of program data, reporting requirements, performance reports and progress reports. This information is often compiled and aggregated by IHS and is subject to a series of internal quality reviews prior to any dissemination to the public. Tribal governments may conduct internal reviews in order to provide data that are accurate, consistent, and complete.

2. Reports, publications, and other products:

   IHS publications are required to be consistent with government-wide and HHS public affairs policies. IHS has guidelines which closely follow the regulations described in the HHS Public Affairs Management Manual and Government Printing and Binding Regulations to assist program staff in obtaining appropriate review and clearance of information products.

3. Statistical data disseminated to the public:

   For IHS demographic and statistical information, IHS uses widely accepted standards for descriptive and analytical statistics to ensure data quality. The IHS statistical program conducts a standard review to ensure data quality, completeness, and reliability. The review includes the reliability of data sources and soundness of statistical methods and presentation.

   Information released by IHS is developed from reliable data sources utilizing accepted methods for data collection. The information is based on thoroughly reviewed analyses and models. The guidelines below describe procedures that IHS employs to assure the quality of its information products, including their utility, objectivity, integrity, transparency, and reproducibility.

4. Utility:

   Utility involves the usefulness of the information to its intended users. Utility is achieved by staying informed of information needs and developing new data, models, and information products where appropriate. Based on internal analyses of information requirements, convening and attending conferences, working with advisory committees, and sponsoring outreach activities, IHS keeps abreast of information needs with respect to the analysis of IHS programs. To the extent possible, we base our analysis on IHS administrative data and surveys by other federal agencies or established survey organizations. When major needs for data related to IHS populations are identified, IHS conducts special-purpose surveys to address these needs or assigns the task to its sister agencies in the Department of Human Services to assist in obtaining the needed data.

   IHS's ongoing publication series and other information products are reviewed to ensure that they remain relevant and address current information needs. Based on internal product reviews, consultation with users, and in response to changing needs and emphases, content of ongoing information products is changed, new products are introduced and others discontinued. IHS prepares special Clinical Trends reports and topical studies that address emerging information needs stemming from data coming from the agency clinical services. It also addresses proposed changes in the law and related policy debates. IHS also identifies requirements for simulation models to support the preparation of analytic reports and policy studies and modifies its current models or develops new models accordingly. Where appropriate, contact information is available on each publication (and in some cases on each table of a publication) to allow feedback and questions by users. In addition, IHS's Office of Program Support has recently begun a periodic customer satisfaction survey to obtain input and guidance by users.

5. Objectivity:

   Objectivity involves a focus on ensuring that information is accurate, reliable and unbiased and that information products are presented in an accurate, clear, complete and unbiased manner. Objectivity is achieved by using reliable data sources and sound analytical techniques, and preparing information products that use proven methods by qualified people that are carefully reviewed.

   Much of the information disseminated by IHS is based on patient data files. These files contain information used to manage IHS programs, including data to determine benefits, and plan for improving the health of the American Indian and Alaska Native people IHS's Office of Public Health conducts ongoing reviews of data and information in IHS data systems to ensure its accuracy. In support of these activities, IHS employs an outside contractor to review IHS's quality control methodology and processes to confirm the validity of its data management processes. IHS administrative data are also covered under IHS Financial Management System and conform to the high standards of financial accountability demanded by these Systems. These financial management systems are mandated by the Office of Management and Budget and are designed to provide complete, reliable, consistent, timely and useful management information to enable agencies to carry out their fiduciary responsibilities.

   IHS-sponsored surveys are conducted using methodologies that are consistent with generally accepted professional standards for all aspects of survey development, including sample frame development, statistical design of the survey sample, questionnaire design and testing, data collection, sampling and coverage errors, non-response analysis, imputation of missing data, weights and variance estimates. IHS surveys follow guidelines and policies set forth in the Paperwork Reduction Act and other regulations related to the conduct of government surveys. IHS also prepares information products using data produced or maintained by other federal agencies and established survey organizations. All such external survey and administrative data used by IHS are produced using generally accepted methodologies.

   Where samples from administrative data files are employed for analysis, sound statistical methods are employed to develop samples. Staff producing statistical publications are knowledgeable about the content, structure and limitations of the administrative data files and maintain working relations with staff who create, update and maintain these files.

6. Use sound analytic techniques:

   Analytical reports are prepared using a variety of analytical techniques from simple tabulations and descriptive summary statistics to multivariate statistical methods and econometric models. Analytical techniques are reviewed for their appropriateness to the data and the analysis being conducted and are clearly identified in reports.

   IHS utilizes several simulation models to make estimates of the effects of demographic and economic trends and legislative and policy options on Indian Health programs and beneficiary populations now and in the future. Some simulation models have been or are being developed within IHS. Others are being developed under contracts. All contracts to develop simulation models provide for detailed documentation that describes the goals and objectives of the model, the data sources being used and the methodologies and assumptions employed. Contract reports are being made available on the Internet. Documentation is available for some simulation models that have been developed within IHS and will be prepared for others.

   All simulation models are extensively tested and reviewed within IHS to verify that the computer programs that were developed to implement the model conform to the stated objectives. Where appropriate, historical simulations are developed to evaluate the success of a model in producing reasonable projections.  Simulation models are based on IHS's and its contractor's best judgments of current and future behavioral relationships and methods of projecting key program outcomes. These models are periodically updated to reflect input from internal and external reviews and research findings on behavioral relationships. These updates are also documented.

7. Preparation of statistical data products:

   Estimates in statistical data products are prepared from representative samples of IHS data files and from reliable external data sources. Procedures for collection of data from administrative files and linking external data files to administrative data files are prepared using accepted statistical methods. Output is reviewed by knowledgeable staff within the originating component and across components as appropriate. Estimates are compared to prior year estimates and estimates from other sources to ensure consistency, reasonableness and reliability. All data sources used in producing statistical data products are identified, either for the publication as a whole or for individual tables. Documentation includes specification of variables used, definitions of variables when appropriate, sampling errors and disclosure avoidance rules or techniques.

8. Preparation of analytical reports and policy studies:

   Information contained in analytical reports and policy studies is based on estimates derived from reliable administrative data files and external data sources. Analysts apply sound statistical and analytical techniques and are knowledgeable about the data sources and models being used. All data sources are identified. When analyses are based on simulation model projections, the assumptions used to produce the projections are also identified as well as the rationale for the assumptions used and the impact of using alternative assumptions. All analytic reports and policy studies are reviewed by technically qualified staff to ensure that analysis is valid, complete, unbiased, objective and relevant. Analytic reports and policy studies that are considered to be more technically complex are also reviewed by subject matter experts outside of the originating component to provide additional perspective and expertise.

9. Editorial review for accuracy and clarity of information in publications:

   All information products are edited and proofread before release to ensure clarity and coherence of the final report. Text is edited to ensure that the report is easy to read and grammatically correct thoughts and arguments flow logically, and information, is worded concisely and lucidly. Tables and charts are edited to ensure that they clearly and accurately illustrate and support points made in the text, and include concise but descriptive, titles. Tables and charts clearly indicate the unit of measure and the universe being examined and all internal labels (column heads, row stubs, and panel headings) should accurately describe the information they contain. All changes made to a manuscript during the editing process are checked by a proofreader and reviewed and approved by the author.

   If an error is detected before an initial mailing, IHS includes an errata notice with the mailing. If the mailing has been sent out, IHS issues an errata sheet with all subsequent publications, and as appropriate, sends the errata sheet to all those who received the initial notice. Errata notices are put on the first page of the Web version to inform both new and repeat site visitors about the mistake, and the corrected version of the document is posted on the Web.

10. Integrity:

    Integrity refers to the security of information from unauthorized access or revision to ensure that the information is not compromised through corruption or falsification. To ensure the integrity of its administrative information, IHS has in place rigorous controls that have been identified as representing sound security practices. IHS is highly protective of the confidentiality of information it holds through its policies and practices. IHS has in place programs and policies for securing IHS resources as required by the Government Information Security Reform Act (P.L. 106-398, Title X, Subtitle G). These security procedures address all major components of information security and apply to all IHS operating components.

    IHS is subject to statutory requirements to protect the sensitive information it gathers and maintains on individuals. These requirements are contained in the following documents:

    • Privacy Act of 1974
    • Computer Security Act of 1987
    • Office of Management and Budget (OMB) Circulars A-123 A-127 and A-130
    • Government Information Security Reform Act
    • Federal Managers' Financial Integrity Act (FMFIA) of 1982
    • Section 6103 of the Internal Revenue Code
    • Section 1106 of the Social Security Act
    • IRS Tax Information Security Guidelines for Federal, State and Local Agencies

VI. Agency Administrative Complaint Procedures

1. Responsibility of the Complainant

   To seek a correction of information disseminated by the IHS, individuals shall follow the procedures described below.

   1. A complaint or request for review and correction of information shall be in written hard copy or electronic form;
   2. it shall be sent to the agency by mail or electronic-mail (e-mail); and
   3. it shall state that an information quality request for correction is being submitted

   The complaint should contain

   4. a description of the specific material that needs to be corrected including where the material is located, i.e. the publication title, date, and publication number, if any, or the website and web page address (url), or the speech title, presenter, date and place of delivery; and
   5. the specific reasons for believing the information does not comply with OMB, HHS, or IHS guidelines and is in error and supporting documentation, if any;
   6. the specific recommendations for correcting the information;
   7. a description of how the person submitting the complaint is affected by the information error; and
   8. the name, mailing address, telephone number, e-mail address, and organizational affiliation, if any, of the individual making the complaint.

   Complainants should be aware that they bear the "burden of proof" with respect to the necessity for correction as well as with respect to the type of correction they seek.

   Complaints by mail should be directed to:

   Indian Health Service (IHS)
   Office of Public Health Support
   5600 Fishers Lane, MS 6NWH04
   Rockville, MD 20857

   Alternatively, complaints may be e-mailed to patrick.weld@ihs.gov and a copy sent to ASPE.InfoQuality@hhs.gov.

2. Responsibility of the Agency

   Based on a review of the information provided, IHS will determine whether a correction is warranted and if, so what action to take. The agency will respond to the requestor by letter or e-mail. The IHS response will explain the findings of the review and the actions that the agency will take, if any. The response will consider the nature and timeliness of the information involved and such factors as the significance of the correction on the use of the information and the magnitude of the correction. The response will describe how the complainant may request reconsideration. The agency will respond to all requests for correction within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.

3. Appeals

   If the individual submitting the complaint does not agree with the IHS's decision (including the corrective action, if any), the complainant may send a written hard copy or electronic request for reconsideration within 30 days of receipt of the agency's decision. The appeal should state the reasons why the agency response is insufficient or inadequate. Complainants should attach a copy of their original request and the agency response to it, clearly mark the appeal with the words, "Information Quality Appeal," and send the appeal to the specific agency appeals address.

   The agency program official who resolved the original complaint will not have responsibility for the appeal. The agency will respond to all requests for appeals within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.

VII. Influential Scientific, Financial and Statistical Information

If an agency is responsible for disseminating "influential" information, guidelines for dissemination should include a high degree of transparency about data and methods to facilitate its reproducibility by qualified third parties. Information is considered influential if it will have a substantial impact on important public policies or important private sector decisions. Since much of IHS's actuarial, statistical and analytical information products potentially have an impact on important public policies, IHS's information that is subject to section 515 should be highly transparent and capable of being reproduced by qualified persons.

IHS's guidelines call for identification and documentation of data sets used in producing estimates and projections and clear descriptions of methods used to produce estimates and to develop model projections to make its results as transparent as possible. Many estimates and projections included in IHS information products are not directly reproducible by the public because the underlying data sets used to produce them are confidential. However, some statistical publications that are based on publicly available data and whose programs are made available on request are fully reproducible by the public. And IHS is in the process of developing public-use versions of several data files, which will increase the reproducibility of estimates and projections to the extent possible while still protecting confidentiality. Some estimates and projections may not be easily reproduced by third parties due to the complexity and detail of the methods and data. In these cases greater emphasis is placed on periodic review by outside panels of technical experts.

IHS also achieves transparency through wide dissemination of its information. Most reports and other data products are available both as printed and electronic documents. They are announced on the IHS web site and most electronic versions can be accessed and downloaded directly from the IHS web site. All documents posted on our Web site since year 2000 are section 508 compliant making information available to an audience that includes persons who have a visual impairment and read online using assistive technology.

NIH MISSION

The National Institutes of Health’s (NIH) mission is to seek fundamental knowledge about the nature and behavior of living systems and the application of that knowledge to enhance health, lengthen life, and reduce illness and disability. Founded in 1887, today NIH is the largest public funder of biomedical research in the world. Consisting of 27 institutes and centers, each with its own mission, approximately 80 percent of NIH's funding is awarded through almost 50,000 competitive grants and research contracts to approximately 300,000 researchers at more than 2,500 universities, medical schools, other research institutions and small businesses in every state and around the world. About 10 percent of NIH's budget supports projects conducted by nearly 6,000 scientists in its own laboratories.

NIH strives to exemplify and promote the highest level of scientific integrity, public accountability, and social responsibility in the conduct of science. NIH has a longstanding commitment to make available to the public the results and accomplishments derived from the activities that it funds. Therefore, NIH-funded intramural and extramural investigators are expected to make the results and accomplishments of their activities available to the research community and the public at large, and to affect their timely transfer to those partners in the health ecosystem that rely on scientific evidence to improve health.

SCOPE OF APPLICABILITY OF INFORMATION QUALITY GUIDELINES FOR NIH

NIH will ensure that disseminated information meets the standards of quality set forth in the Office of Management and Budget Information Quality Guidelines (OMB Guidelines), the Department of Health and Human Services Guidelines for Ensuring the Quality of Information Disseminated to the Public (HHS Guidelines), and the NIH Information Quality Guidelines (NIH Guidelines). It is NIH's goal to ensure and maximize the quality, objectivity, utility, and integrity of information that it disseminates to the public. NIH strives to provide information that is accurate, reliable, clear, complete, unbiased, and useful and is committed to integrating the principle of information quality into every phase of information development, including creation, collection, maintenance, and dissemination.

The pre-dissemination review described in the NIH Guidelines only applies to official information (with the NIH imprimatur) released on or after October 1, 2002. The administrative mechanism for correction applies to information that the agency disseminates on or after October 1, 2002, regardless of when the agency first disseminated the information.

The NIH Guidelines apply to information in all media—printed, electronic, audiovisual, verbal, and other. They focus primarily on the dissemination of substantive information (i.e., reports, studies, summaries) rather than information pertaining to basic agency operations. Information that is disseminated at the request of NIH or with specific NIH approval through a contract or a grant is subject to these NIH Guidelines. The OMB Guidelines identify the types of information that are and are not covered. Examples of that guidance as applied to NIH information are provided below. Although information that is not covered by the OMB Guidelines is not subject to the administrative complaint procedures, the information is still subject to the usual NIH internal review procedures for accuracy and quality.

NIH Information Covered by the OMB Guidelines

• Scientific research papers, books, journal articles, and similar authoritative materials produced by NIH employees and contractors, unless they have disclaimers alerting the audience that they do not represent official views of NIH. (This does not apply to materials produced by extramural researchers supported by NIH awards.)
• Other official reports, brochures, documents, newsletters, fact sheets, and audiovisual productions (i.e., a unified presentation, developed according to a plan or script, containing visual imagery, sound, or both, and used to convey information).
• Editorials, commentaries, letters-to-the-editor, only if written by NIH staff representing official NIH viewpoints.
• Oral information, including speeches, interviews, expert opinions, only if representing NIH's views, official positions, or policies.
• Open meetings' proceedings and minutes.

NIH Information Not Covered by the OMB Guidelines

• National Library of Medicine (NLM) databases or other archival records, grant information contained on the Research Portfolio Online Reporting Tools (RePORT), and similar databases.
• Documents not authored by the agency and not representing the agency's views, including information authored and distributed by NIH grantees.
• Information that is limited in dissemination to government employees or agency contractors or grantees.
• Information pertaining to basic agency operations.
• Responses to requests for agency records under the Freedom of Information Act (FOIA), the Privacy Act, the Federal Advisory Committee Act, or other similar laws.
• Information relating solely to correspondence with individuals or persons.
• Press releases that support the announcement or give public notice of information that NIH has disseminated elsewhere.
• Information intended for public filings, subpoenas, or adjudicative processes.
• Opinions where the agency's presentation makes it clear that what is being offered is personal opinion rather than fact or the agency's views.

TYPES OF INFORMATION DISSEMINATED BY NIH TO THE PUBLIC

Each year, NIH components produce a multitude of communications published electronically and also in hard copy. All publications that carry the NIH imprimatur, i.e., are considered official NIH publications or releases, must follow NIH policy and procedures for preparation, review, approval, and distribution (see Section V). The types of information disseminated by NIH to the public include the following:

Scientific Reports

Reports presenting scientific findings that may be in the form of a book, chapter of a book or textbook, monograph, journal article, proceedings, generally authored or co-authored by NIH staff as part of their official duties or working groups convened by NIH. This includes authoritative health information derived from the deliberations of expert panels.

Program Reviews, Analyses, and Evaluations

Reports summarizing the results of program assessments and research syntheses used for meeting annual reporting requirements and/or for communicating program and policy directions.

Policies, Guidelines, and Funding Opportunity Announcements

Policies and guidelines for the conduct of NIH-supported research to ensure research integrity and regarding the availability of targeted funding. The NIH Guide for Grants and Contracts (https://grants.nih.gov/funding/nih-guide-for-grants-and-contracts) is NIH's official publication of notices of grant policies, guidelines, and other information.

Statistical Data

Primarily administrative data, including information concerning NIH’s extramural and intramural research portfolio.

Editorials, Commentaries, Letters-to-the-Editor

Authored by NIH staff representing official NIH viewpoints.

Consumer Information

Resources for the general consumer that include information about health conditions and their diagnosis, prevention, and treatment.

Science Education Materials and Training Modules

Materials geared towards all levels of education as well as training modules for researchers.

Press Releases 

NIH press releases that communicate noteworthy NIH information such as research advances and consumer health news.

TYPES OF DISSEMINATION METHODS

NIH information is disseminated in many mediums, with the following four being most common:

Print

Publications, books, newsletters, brochures, booklets, pamphlets, and reports.

Oral

Formal speeches, oral presentations, interviews, or commentaries for publication or broadcast.

Audio-Visual

Broadcast scripts, podcasts, and videocasting.

Electronic

The NIH website (https://www.nih.gov/) is not just one site but a large collection of sites, with over 150 servers.

AGENCY QUALITY ASSURANCE POLICIES, STANDARDS, AND PROCESSES FOR ENSURING THE QUALITY OF INFORMATION DISSEMINATED TO THE PUBLIC

All NIH communications are prepared in accordance with professional and ethical standards and must be appropriate for dissemination after undergoing appropriate review and approval prior to release. NIH adheres to the laws and regulations applying to communications, including the OMB Guidelines, the HHS Guidelines, and NIH Manual Chapter 1184: Preparation and Clearance of Scientific, Technical, and Public Information Presented by NIH Employees or Produced for Distribution by NIH, last updated in March 2017. Efforts to ensure and maximize information quality begin at the preparation stage and continue through the review and approval stages. Existing NIH policies developed in concert with federal electronic security laws provide appropriate security safeguards to ensure integrity of NIH documents, so that the information is protected from unauthorized access, revision, corruption, or falsification. Information to be published on the public-facing NIH website must adhere to NIH Manual Chapter 2804: Public-facing Web Management Policy.

The review procedures may vary depending upon the requirements for the type of information to be disseminated:

• Scientific Information Communicated in Journal Articles and Meeting Presentations. The presentation of any scientific research results or other professional findings is typically made by publication in a scientific or professional journal or presentation at a meeting of a professional organization. The choice of the journal or meeting to which reports are offered is the prerogative of the author(s). Any manuscript prepared by an NIH employee on a subject related to his/her NIH duties must be reviewed and approved through an internal NIH process prior to submitting it for publication consideration. In addition, most publications in scientific and professional journals are customarily subjected to an external peer review process directed by the interested publisher, volume editor, or journal editor. With few exceptions, oral presentations on scientific information, health policy or practice, or presentations with policy implications, must also be cleared in advance through internal processes within each NIH Institute, Center, or Office.
• Program Reviews, Analyses, and Evaluations. Reports summarizing the results of program assessments and research syntheses used for meeting annual reporting requirements and/or for communicating program and policy directions are subject to internal reviews by several agency officials for quality, objectivity, and accuracy. Federal experts from within and outside of NIH often monitor interim and final reports of projects with crosscutting implications. Information contained in reports is based on estimates derived from reliable administrative data. All data sources are identified. All analytic reports are reviewed by technically qualified staff to ensure valid, complete, unbiased, objective, and relevant analysis.
• Policies, Guidelines, and Funding Opportunity Announcements. The NIH Guide for Grants and Contracts (https://grants.nih.gov/funding/nih-guide-for-grants-and-contracts) is NIH's official publication of notices of grant policies, guidelines, and other information. All Guide notices are cleared through NIH and Departmental clearance procedures, if applicable, as well as reviewed by numerous other subject matter experts within the NIH Office of the Director.
• Statistical Data. Administrative data including information concerning NIH’s extramural and intramural research portfolio are posted to the web following procedures to assure the quality, integrity, and consistency of the data.
• Editorials, Commentaries, Letters-to-the-Editor. Any statements, comments, or discussion of federal policies or practices or NIH scientific findings that are relevant to the employee's position or duties, draw conclusions, advocate or oppose professional practices or positions on subjects related to NIH duties, or might otherwise be construed as reflecting an official position by NIH, HHS, or the Federal Government, must be approved by NIH leadership.
• Consumer Information. As part of the agency’s congressionally mandated public education activities, new health and medical findings often represent an imperative for NIH Institutes and Centers to develop and release informational products for wide public dissemination. Almost without exception, these products and materials are linked to agency-sponsored research programs and can include products such as booklets, fact sheets, newsletters, pamphlets, bookmarks, web content, and health posters. All products and materials are reviewed by Institute and Center leadership to ensure they are of high quality (i.e., are produced with objectivity, utility, and integrity), are accurate, both in specific details and in general, and are 508 compliant.
• Report on Carcinogens. This congressionally mandated report lists substances in our environment that may be cancer hazards for persons residing in the United States. It is prepared by the National Institute of Environmental Health Sciences as part of its activities for the interagency National Toxicology Program (NTP). The NTP is an interagency partnership of NIEHS, the National Institute for Occupational Safety and Health, and the Food and Drug Administration, and is located administratively at NIEHS. NIEHS/NTP follows an established process to develop recommendations regarding the addition to, or removal of, or modification to, a substance listing in the report that is transmitted to the HHS Secretary for approval.

NIEHS/NTP procedures conform to accepted NIH scientific practices where quantitative and qualitative scientific conclusions are based on: (1) the best available science and supporting studies, particularly peer-reviewed studies, conducted in accordance with sound and objective scientific practices; and (2) data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies use of the data). NIEHS/NTP makes every effort to ensure that the presentation and dissemination of information about environmental health is comprehensive, informative, and understandable.

AGENCY ADMINISTRATIVE COMPLAINT PROCEDURES

Responsible Official

NIH has developed administrative mechanisms to allow affected persons to seek and obtain correction of disseminated information that does not comply with OMB, HHS, and NIH Guidelines.

The NIH official with overall responsibility for receiving and resolving complaints regarding information that does not comply with agency guidelines for information quality is the Director of the Office of Evaluation, Performance, and Reporting (OEPR) in the Division of Program Coordination, Planning, and Strategic Initiatives, Office of the Director, NIH. The Director of OEPR has overall responsibility for implementing NIH Information Quality Guidelines and works collaboratively with other NIH units.

Responsibility of the Complainant

To seek a correction of information disseminated by NIH or its components, an individual should email the request to OEPR_InfoQuality@mail.nih.gov or mail the complaint to:

Director, Office of Evaluation, Performance, and Reporting
Division of Program Coordination, Planning, and Strategic Initiatives
Office of the Director
National Institutes of Health 
Building 1, Room 260
1 Center Drive
Bethesda, MD 20892
 

The request should state that an information quality request for correction is being submitted, and should provide the following information:

• A detailed description of the specific material that is proposed for correction, including where the material is located, i.e., the publication title, date, and publication number, if any, or the website and web page address (URL), or the presentation, presenter, date, and mode of delivery.
• The specific reasons for believing that the information does not comply with OMB, HHS, or NIH Guidelines and is in error, and supporting documentation, if any.
• Suggested recommendations for what corrective action(s) should be taken.
• A description of how the person requesting the correction is affected by the information error.
• Complete contact information for the requestor, including name, mailing address, telephone number, e-mail address, and organizational affiliation, if any.

Complainants should be aware that they bear the "burden of proof" with respect to the necessity for correction as well as with respect to the type of correction they seek.

Responsibility of the Agency

Based on a review of the information provided, NIH will determine whether a correction is warranted and if so, what action to take. The agency will respond to the requestor by letter or e-mail. The agency's response will explain the findings of the review and the actions that the agency will take, if any. The response will consider the nature and timeliness of the information involved and such factors as the significance of the correction on the use of the information and the magnitude of the correction. The response will describe how the complainant may request reconsideration. The agency will strive to respond to all requests for correction within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.

Appeals

If NIH denies a request for correction, the complainant may send a written request for reconsideration within 30 days of receipt of the agency's decision. The request should state the reasons for the appeal and requestors should reference the NIH tracking number provided in the NIH response to the original request. Requests may be sent as e-mail to OEPR_InfoQuality@mail.nih.gov or as hard copy to the following address:

Director, Office of Evaluation, Performance, and Reporting
Division of Program Coordination, Planning, and Strategic Initiatives
Office of the Director
National Institutes of Health 
Building 1, Room 260
1 Center Drive
Bethesda, MD 20892
 

If sent by hard copy, requestors should clearly mark the outside envelope, "Information Quality Appeal."

NIH will strive to respond to all requests for appeals within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason for the delay and an estimated decision date.

INFLUENTIAL SCIENTIFIC, FINANCIAL, AND STATISTICAL INFORMATION

The OMB Information Quality Guidelines require that "influential scientific, financial, or statistical information” in official government documents must be based on studies that can be substantially reproduced if the original or supporting data were independently reanalyzed using the same methods. "Influential" when used in the phrase "influential scientific, financial, or statistical information" means that NIH can reasonably determine that dissemination of the information will have or does have a clear and substantial impact on important public policies or important private sector decisions, or will have important consequences for specific health practices, technologies, substances, products, or firms. NIH is committed to applying rigorous scientific standards to ensure the accuracy, reliability, and reproducibility of research results.

The reproducibility standard applies to analytic results and not necessarily to the original and supporting data used to produce the analytic results. To facilitate the replication of scientific and other influential information by qualified third parties, NIH encourages the sharing of scientific data and methods where practicable. After publication, supporting data that form the basis of any research communication and do not constitute any disclosure risk, should be made available promptly and completely to any person who seeks further information. Any unique reagents and sensitive data should be made available to any qualified person who seeks further information.

Since the influence and implications of NIH-disseminated information cannot always be fully anticipated, all NIH scientific reports are expected to state clearly how analytical results are generated—the specific data used, various assumptions, specific analytic methods, statistical procedures, sources of error—making the analysis sufficiently transparent so as to be capable of being reproduced. NIH advocates the archiving of data where feasible to facilitate the reproducibility of influential information. Exceptions may be necessary to maintain the privacy, rights, and confidentiality of human participants’ data or if unique materials were developed or obtained under agreements that preclude their dissemination. Investigators should retain research data as long as it is deemed useful to the research community or public. In situations where public access to underlying data is not practicable, NIH shall apply especially rigorous robustness checks to analytic results and document what checks were undertaken.

For scientific and technical documents, the scientific community recognizes peer review as the primary means of quality control. NIH routinely seeks input from qualified peer reviewers of influential materials for propriety, accuracy, completeness, and quality (including objectivity, utility, and integrity) prior to dissemination. Although concerted efforts are made to ensure that influential information is subjected to rigorous peer review and reproducibility specifications, standard operating procedures may be temporarily disrupted under urgent situations, such as when an imminent threat to public health or homeland security is identified.

NIH makes every effort to ensure that the presentation and dissemination of information is comprehensive, informative, and understandable, and that scientific conclusions are based on: (1) the best available science and supporting studies, particularly peer-reviewed studies, conducted in accordance with sound and objective scientific practices; and (2) data collected by accepted methods or best available methods (if the reliability of the method and the nature of the decision justifies use of the data).

SPECIAL CONSIDERATIONS FOR AGENCY DISSEMINATION

Sometimes NIH provides "late breaking news" of urgent import to the public on research findings prior to publication in scientific journals or reports and prior to peer review. Under such special circumstances, NIH may temporarily waive adherence to the information quality guidelines. However, when it does so, there is an internal review process that routinely draws upon external expertise and relevant monitoring or advisory boards to ensure that information disseminated to the public summarizes the facts as NIH currently knows them, and that appropriate disclaimers are attached.

REFERENCES

NIH Manual Chapter 1184 – Preparation and Clearance of Scientific, Technical, and Public Information Presented by NIH Employees or Produced for Distribution by NIH, March 24, 2016, https://policymanual.nih.gov/1184#A2648C1F

Office of Management and Budget Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal Agencies. Final Guidelines. February 22, 2002 https://www.federalregister.gov/documents/2002/02/22/R2-59/guidelines-for-ensuring-and-maximizing-the-quality-objectivity-utility-and-integrity-of-information

OMB, Circular No. A-130, Revised (Transmittal Memorandum No. 4). Management of Federal Information Resources. November 30, 2000.

I. Agency Mission

The Substance Abuse and Mental Health Services Administration (SAMHSA) was established by Congress under Public Law 102-321 on October 1, 1992, to strengthen the Nation's health care capacity to provide prevention, diagnosis, and treatment services for substance abuse and mental illnesses. SAMHSA works in partnership with States, communities and private organizations to address the needs of people with substance abuse and mental illnesses as well as the community risk factors that contribute to these illnesses.

SAMHSA serves as the umbrella under which substance abuse and mental health service Centers are housed, including: the Center for Mental Health Services (CMHS), the Center for Substance Abuse Prevention (CSAP), and the Center for Substance Abuse Treatment (CSAT). SAMHSA also houses the Office of the Administrator, the Office of Applied Studies, and the Office of Program Services.

II. Scope and Applicability of Guidelines for SAMHSA

1. Overview

   SAMSHA will ensure that disseminated information meets the standards of quality set forth in the OMB, HHS and SAMSHA guidelines. It is SAMSHA’s policy to ensure and maximize the quality, objectivity, utility, and integrity of information that it disseminates to the public. We strive to provide information that is accurate, reliable, clear, complete, unbiased, and useful. We are committed to integrating the principle of information quality into every phase of information development, including creation, collection, maintenance, and dissemination.

   The pre-dissemination review described in the guidelines only applies to information disseminated on or after October 1, 2002. The administrative mechanism for correction applies to information that the agency disseminates on or after October 1, 2002, regardless of when the agency first disseminated the information.

   They apply to information in all media - printed, electronic, audiovisual, verbal, and other. The Guidelines focus primarily on the dissemination of substantive information (i.e., reports, studies, summaries) rather than on the information pertaining to basic agency operations. Information that is disseminated at the request of SAMHSA or with specific SAMHSA approval through a contract or grant is subject to these Guidelines.

2. Covered Information

   Scientific research papers, books, journal articles, and similar materials unless they have disclaimers alerting the audience that they do not represent official views of SAMHSA;

   Other official reports, brochures, documents, newsletters, electronic documents, and audiovisual productions (i.e., a unified presentation, developed according to a plan or script, containing visual imagery, sound, or both, and used to convey information);

   Oral or written information, including speeches, interviews, expert opinions, if representing SAMHSA's official positions, views, or policies;

   Statistical information, including statistical analyses, aggregated information by program, Center or Office, or for SAMHSA as a whole, including funding information and histories (by funding mechanism, dollars, and/or other criteria);

3. Information Not Covered
   1. Documents not authored by the agency and not representing agency's views, including information authored and distributed by SAMHSA grantees¹
   2. Information that is limited in dissemination to Government employees or agency contractors or grantees;
   3. Information pertaining to basic agency operations, e.g., information about agency authorities, activities, programs, along with contact information for the public, organizational charts, SAMHSA or Center Directors' Status Reports, solicitations (for grants or contracts), receipt and review materials (e.g., summary statements, information for advisory councils or advisory committee members);
   4. Information intended solely for intra-or inter-agency use or sharing of Government information;
   5. Responses to requests for agency records under the Freedom of Information Act (FOIA), the Privacy Act, the Federal Advisory Committee Act, or other similar laws;
   6. Information relating solely to correspondence with individuals or persons;
   7. Press releases that support the announcement or give public notice of information that SAMSHA has disseminated elsewhere.
   8. Information for public filings, subpoenas, or adjudicative processes;
   9. Opinions where the agency's presentation makes it clear that what is being offered is personal opinion rather than fact or the agency's views.

III. Types of Information Disseminated

All publications that carry the SAMHSA imprimatur, i.e., that are considered official SAMHSA publications or releases, must follow SAMHSA policy and procedures for preparation, review, approval, and distribution. The types of information disseminated by SAMHSA to the public include the following, however, the OMB guidelines are not directly applicable to all the information in these categories (See Section II):

Scientific Research Papers, Books, Journal Articles, etc. These may be in the form of a book, chapter of a book or textbook, monograph, journal article, proceedings, or the like. These are generally authored or co-authored by SAMHSA staff as part of their official duties and may be authored by working groups convened by SAMHSA. This category also includes grant project description (e.g., abstracts of funded grant proposal available through the NIH Computer Retrieval of Information on Scientific Projects database), bibliographies, collections of abstracts, reviews, and recurring reports. Summaries of grant findings may be released in the form provided by the investigator, or the investigator-supplied information may be used as the basis of a narrative describing a particular program area. These summaries may be used for many purposes, including in meeting annual reporting requirements, such as the Government Performance and Results Act (GPRA). Highlights of findings may be posted on the SAMHSA web site and may be found in testimonies and speeches by SAMHSA staff in many venues, including annual Appropriations Hearings, presentations on SAMHSA funding opportunities, and literature reviews.

Official Reports, Brochures, Documents, and Audiovisual Productions. This category includes:

Summaries and Analyses of Data Collection Activities. Periodically, SAMHSA publishes summaries of its data collection activities, such as the annual Summary of Main findings from the National Household Survey on Drug Abuse and preliminary estimates from the Drug Abuse Warning Network.

Evaluation and Program Findings. SAMHSA sponsors a variety of evaluation and other activities which lead to major publications such as the recently completed National Treatment Improvement Evaluation Study. SAMHSA also publishes information derived from its grant and contract sponsored programs such as the National Estimates of Expenditures for Mental Health and Substance Abuse Treatment.

Grant and Contract Funding Opportunities. The SAMHSA web site is the official vehicle for announcing the availability of SAMHSA funds for discretionary grants and contracts. Current and past SAMHSA funding opportunities are available on the SAMHSA web site, along with other information on grants policy, peer review, award data, contacts, application forms, and CRISP database.

Guidelines. This type of information is issued after careful review and deliberation of available scientific evidence, usually with the assistance of a panel of outside experts and is generally associated with a formal meeting or consensus panel specifically convened for the purpose. Prime examples are the Treatment Improvement Protocols published by the Center for Substance Abuse Treatment, SAMHSA.

Oral or Written Information. This category includes official speeches, editorials, commentaries, Letters-to-the-Editor only if they are provided by SAMHSA staff representing official SAMHSA viewpoints.

Statistical Information. Examples of statistical information published by SAMHSA include annual appropriations by SAMHSA's Centers or Offices, employment data (e.g., numbers of staff and staffing by professional degree), and data books (e.g., the annual Summary of Findings for the National Household Survey on Drug Abuse).

IV. Types of Dissemination Methods

SAMHSA information is disseminated in many media, with the following four being the most common:

Print -- publications, books newsletters brochures, booklets, pamphlets, and reports.

Oral -- formal speeches, oral presentations, interviews, or commentaries for publication or broadcast; letters to the editor or correspondence likely to result in similar publications.

Audio-Visual -- broadcast scripts, audio or videotapes, and videocasting.

Electronic -- The SAMHSA web site at www.samhsa.gov.

V. Quality Assurance Policies, Standards, and Processes for Ensuring the Quality of Information Disseminated to the Public.

1. Overview

   At SAMSHA, the quality assurance process begins at the inception of the information development process. Further, SAMSHA reviews the quality (including the objectivity, utility, and integrity) of information before its is disseminated and treats information quality as integral to every step of the development of information, including its creation, collection, maintenance and dissemination.

   All SAMHSA documents and audiovisuals must be prepared in accordance with professional and ethical standards, as well as generally accepted standards of good taste. They must be appropriate for dissemination by SAMHSA and must undergo appropriate review and approval prior to release. SAMHSA adheres to the laws and regulations applying to publications and audiovisual materials, including OMB Information Quality Guidelines, the HHS Printing Handbook, and relevant SAMHSA policy issuances. SAMHSA efforts to ensure and maximize information quality begin at the preparation stage and continue through the review and approval stages. When published electronically, existing SAMHSA policies developed in concert with Federal computer security laws provide appropriate security safeguards to ensure integrity of SAMHSA documents, i.e., that the information is protected from unauthorized access, revisions, corruption, or falsification. Further, SAMHSA is committed to demonstrating in its Paper Reduction Act (PRA) will be collected, maintained, and used in a way that is consistent with OMB, HHS and SAMSHA information quality guidelines.

2. Preparation

   Each publication must be accurate, both in specific details and in general impressions, and meet accepted standards of high quality. The OMB Information Quality Guidelines define quality as including objectivity, utility, and integrity. SAMHSA documents and presentations containing text and summary data must be objective and scientifically sound. Sources should be referenced for the convenience and further information of the reader. Where appropriate, supporting data should have full, accurate, and transparent documentation. Potential error sources affecting data quality should be identified and disclosed to users. Disclaimers should be used to distinguish the status of information (e.g., based on preliminary data or partial data set).

   OMB Information Quality Guidelines hold "influential" scientific, financial, or statistical information to a higher standard of quality, requiring that the results be "substantially reproducible" by qualified third parties were the original or supporting data to be independently re-analyzed using the same methods. Making the data and methods publicly available will assist in determining whether analytical results are capable of being substantially reproduced.

3. Plain Language

   A Government-wide directive requires Federal agencies to use plain language in all communications with the public. Plain language is writing that is geared to the target audience (i.e., a plain language document for a scientific audience may be different from a plain language document for the general public). Plain language is grammatically correct, with accurate word usage. It is clear and expresses exactly what readers need to know without unnecessary words.

4. Layout and Design

   Each publication should be prepared in a pleasing, dignified, and finished format appropriate for its intended use and audience. Use of color and typography must be in accord with Government Printing and Binding Regulations (www.house.gov/jcp/jcpres.pdf).

5. SAMHSA Information Review and Approval Policies and Procedures by Type of Information

   1. Overview

      The purpose of the review process is to improve the quality of SAMHSA documents and to ensure the accuracy and validity of information intended to benefit the targeted audience, such as professional associations and the general public. All materials distributed by SAMHSA must be reviewed for accuracy, propriety, completeness and quality (including objectivity, utility, and integrity). The structure of the review and the types of reviewers will depend on the nature of the information as well as the targeted audience. For scientific and technical documents, peer review provides a level of quality control that is well recognized in the scientific community. According to OMB Information Quality Guidelines, material subject to formal, independent, external peer review can be considered to be of acceptable objectivity. However, this presumption of objectivity is refutable based on a persuasive showing by a complainant in a particular instance. The single most important determinant of a review groups' excellence and credibility are its members. Reviewers must: have appropriate scientific knowledge (as demonstrated by grant and publication records, academic degrees and honors); be respected in the scientific community; have breadth of expertise; be fair and objective; and, not be influenced by inappropriate personal interests (competition, scientific bias, personal antagonisms, or similar irrelevant factors). Because utility is one measure of quality, the review group should include representatives of the targeted audience to the degree practicable.

      Statistical compendia and documents providing "influential scientific, financial, or statistical information" must be reviewed carefully. The Director of the Office of Communications must determine that these documents and the data on which they are based conform to the standards set forth here, and, if applicable, that the reported information and/or statistics reported therein be substantially reproducible.

      A document that has obtained publication clearance for paper printing is often posted on the sponsoring Center's or Office's web page for greater accessibility. Such documents should not need additional approvals. Web documents with no print counterpart require content clearance by the Office of Communications to ensure that the information observes all applicable requirements governing information for release to the public.

      Scientific papers, books, journal articles, brochures, documents, statistical compendia, newsletters, electronic documents, audio-visual productions, and similar materials.

      SAMHSA encourages dissemination of scientific research and other information by its employees. Professional and scholarly writing, lecturing, editing, and publishing are an essential part of research, are in the public interest, and bring credit and distinction to SAMHSA and to the employees themselves. In assisting employees to share information about their official and professional activities, SAMHSA seeks to advance scientific knowledge and contribute to professional education. The first report of any scientific research results or other professional findings may be made by publication in a scientific or professional journal or presentation at a meeting of a professional organization. The choice of the journal or meeting to which reports are offered is the prerogative of the author(s).

      To ensure and maximize the quality of information disseminated by SAMHSA employees, any non-extemporaneous presentation (written or electronic) by a SAMHSA employee on a subject related to his/her SAMHSA duties must be reviewed and approved through an internal SAMHSA process prior to submitting for publication consideration. With few exceptions, non-extemporaneous oral presentation on health policy or practice, or presentations with policy implications, must also be cleared in advance. Manuscripts intended for publication are customarily subjected to an external peer review process directed by the interested publisher, volume editor, or journal editor.

      Publication or oral presentation of scientific and professional information by individual employees must conform to applicable laws and regulations, including OMB Information Quality Guidelines and the HHS Standards of Conduct Regulations.

      Customary professional practices impose certain constraints on the degree to which SAMHSA employees maybe identified with the results of research and development work, including that obtained in collaboration with extra-mural grantees.²

      All SAMHSA components have a formal internal operating procedure for identifying printing requirements and tracking publications. In addition, the concept clearance process for new publications is often the vehicle to track the development of a new publication and to identify its attendant printing requirements. Each prospective publication must be cleared through the originating SAMHSA component and then be approved by the Office of Communications, SAMHSA and the Office of the Assistant Secretary for Public Affairs (OASPA), HHS.

      In brief, any official publication (including book, bibliography, chapter of a book or textbook, booklet, brochure, collection of abstracts, fact sheet, house organ, index, leaflet, manual, monograph, newsletter, pamphlet, review, periodical, proceeding, recurring report, statistical compendium, Internet document, audio-visual or the like) prepared by any SAMHSA component directly or through a contract must be sent by SAMHSA for HHS clearance. This clearance requirement does not apply to publication of articles in journals.

      All audio-visual projects and exhibits must be cleared through OASPA, whether produced in-house or under contract. To obtain clearance for audiovisual products, including exhibits, Form HHS 524A must be completed, approved by the Office of Communications, and filed and approved with the Office of the Assistant Secretary for Public Affairs before production may begin.

      Statistical compendia, including statistical analyses, aggregated information by program, Center, Office or for SAMHSA as a whole, including funding information and histories (by funding mechanism, dollars, or other criteria), require approval by the Director, Office of Communications, SAMHSA. The Director of the originating Center or Office is required to determine that the data conform to accepted scientific and quality standards and that the reported statistics are substantially reproducible.

      In general, any writing by a SAMHSA employee on a work-related subject, whether or not intended for electronic or print publication, or for oral delivery, must be prepared according to accepted SAMHSA standards of quality, reviewed for substantiative content, and administratively approved. The purpose of the clearance process is to improve the quality of information and to ensure the accuracy, objectivity, utility, and validity of information.

      Consistent with HHS Standards of Conduct (73.735-705 Writing and Editing), employees are encouraged to engage in outside writing and editing when such activity is not otherwise prohibited. If the writing or editing activity is related to the employee's official duties or other responsibilities and programs of the Federal Government, the employee must: (1) make no mention of his or her official title or affiliation with the Department, or (2) use his or her official title or affiliation with the Department, or (3) submit the material for clearance within the operating component, under procedures established by the component. When clearance is denied at any lower level, the employee shall have recourse for review up to the head of the principal operating component. This clearance will show there are no official objections to the activity, and the employee may then use his or her official title or affiliation usually without a disclaimer. Except where the requirement for disclaimer is waived as a result of official clearance, disclaimers shall be used in all writing and editing related to the employee's official duties or other responsibilities and programs of the Federal Government: (1) in which the employee identifies himself or herself by official title or affiliation with the Department, or (2) when the prominence of the employee or the employee's position might lead the public to associate him or her with the Department, even without identification other than by name. Disclaimers shall read as follows unless a different wording is approved by the Assistant General Counsel, business and Administrative Law Division, Office of the General Counsel: "This (article, book, etc.) was (written, edited) by (employee's name) in (his or her) private capacity. No official support or endorsement by (name of operating component or of Department) is intended or should be inferred."

      Oral Information including speeches, interviews, expert opinions, only if representing SAMHSA views, official position, or policies

      Any public statements, comments, or discussion of Federal policies or practices that are relevant to the employee's position or duties, draw conclusions, advocate or oppose professional practices or positions on subjects related to SAMHSA duties or that might otherwise be construed as reflecting an official position by SAMHSA, HHS, of the Federal Government, are covered by the OMB Guidelines and must be approved in the Office of Communications, SAMHSA.

      No review or approval is required for non-official and private writing, speaking, and publishing by an employee unless his/her SAMHSA employment is likely to be regarded as influencing the content.

      SAMHSA employees are responsible for the statements they make, regardless of whether or not they have been cleared. If one presents material that requires clearance but that has not been cleared prior to presentation, then the employee must inform the audience of the personal or unofficial nature of his or her views. An example of an appropriate disclaimer follows:

      "This material is presented from my own perspective and should not be taken as representing the viewpoint of the Department, SAMHSA, or the Center for...."

      SAMHSA employees shall not identify themselves as SAMHSA employees in unofficial materials prepared for dissemination to non-professional audiences, such as a letter to the editor. These materials must be reviewed prior to presentation in the Office of Communications, if the employee's identification with SAMHSA is to be shown, can be inferred, or is well known.

      SAMHSA Clearinghouse Information

      Clearinghouses often serve as a public point of contact and provide access to information about SAMHSA programs, conferences, and grant activities. These are often in electronic as well as hard copy format. Clearinghouses have been contracted to provide varying levels of service, including distribution of fact sheets, information packages, and publications; storage of materials; conducting outreach and promotion; and performing training and quality control for the clearinghouse staff. Clearinghouses may respond to inquiries about particular issues, ranging from information about available educational materials to statistical data. Clearinghouses are challenged to ensure accuracy and reliability of information, while continually striving to improve performance and response times. Clearinghouses also must determine which organizations are worthy of referral when customers need information that is not available at the clearinghouse and how to avoid implying endorsement.

   2. Procedures to Ensure the Integrity of Information.

      SAMHSA has developed World Wide Web (www) Guidance which is available on the SAMHSA Intranet to SAMHSA staff. Web page creators must periodically review material on the web page to determine whether or not it is accurate and up-to-date. Information, particularly time-sensitive information, should be posted as soon as possible. Web page creators are expected to promptly update or remove out-of-date information or to notify Division of Information Resources Management that such actions must be taken.

      Unless otherwise noted, it is safe to assume that information posted on public web sites within the "SAMHSA.gov" domain is considered to be "in the public domain." As such, others are free to establish links to SAMHSA online resources. In establishing such links, SAMHSA requests that others avoid creating the impression that SAMHSA is endorsing or promoting any particular product or service. In the same vein, any outside link to an external resource from a SAMHSA web site needs to be examined and approved on a case-by-case basis.

      Web pages containing links to external pages not located on the SAMHSA server should include a link to a statement that releases SAMHSA from responsibility for the material included in the external web page. Again, it is important to avoid giving a user the impression that SAMHSA is endorsing information or a commercial product described in an external site. Disclaimers on copyright, endorsement (general and external links), liability, and medical information are also used, as appropriate, for individual web sites.

      Each Center or Office designates a principal contact for information and approvals related to the development of web pages. SAMHSA personnel, contractors, and other authorized users must notify this contact person prior to setting up a web page on the SAMHSA server.

      The SAMHSA Division of Information Resources (DIRM) is charged with providing, coordinating, and managing information technology for SAMHSA. In terms of computer security, there are three distinct objectives: (i) confidentially -- ensuring that there is no deliberate or accidental improper disclosure of sensitive automated information; (ii) integrity -- protecting against deliberate or accidental corruption of automated information; and, (iii) availability -- protecting against deliberate or accidental actions that cause automated information resources to be unavailable to users when needed. Information is accorded protection against disclosure, alteration, loss, or destruction, based on the degree of sensitivity.

      DIRM staff use appropriate safeguards to protect data from improper disclosure by backing up critical data periodically, and, if a security incident occurs, by following proper incident response procedures. Supervisors are responsible for ensuring that employees, both Government and contractors observe all security requirements and that employees receive appropriate security training.

VI. Agency Administrative Complaint Procedures

SAMSHA has developed administrative mechanisms to allow affected persons to seek and obtain correction of disseminated information that does not comply with OMB, HHS and SAMSHA guidelines.

At SAMHSA, the Director, Office of Communications (OC) will have overall responsibility for implementing SAMHSA's Information Quality Guidelines and coordinating this effort with other SAMHSA Centers and Offices. All complaints may be sent to:

Substance Abuse and Mental Health Services Administration (SAMHSA)

Center for Behavioral Health Statistics and Quality
Substance Abuse and Mental Health Services Administration
5600 Fishers Lane
Rockville, MD 20857

Email: cbhsqrequests@samhsa.hhs.gov

OC is the central office for communications at SAMHSA. As such, OC takes the lead across SAMHSA for setting communications policy and for communicating information about SAMHSA programs, issues, and accomplishments to the public, public interest groups, and to the scientific community. OC is the communications link between SAMHSA's components and the Office of the Assistant Secretary for Public Affairs (OASPA) in HHS and serves as the coordinating office or central source for matters related to SAMHSA publications, including printing, HHS/SAMHSA clearance and review procedures, Joint Committee on Printing, U.S. Congress, and Government Printing Office printing and binding regulations, and copyright rules. Among its many activities, the office produces and distributes a number of publications that highlight SAMHSA achievements. OC also support and coordinates the principal contents of the SAMHSA web site (www.samhsa.gov).

As the lead office for SAMHSA Information Quality, OC responsibilities include:

Developing policies and procedures to effectively meet the requirements of the OMB Information Quality Guidelines;

Providing information and/or training to SAMHSA staff on their responsibility in meeting Federal requirements and SAMHSA policies on ensuring the quality of information disseminated to the public;

Assisting in the review of information quality complaints;

Reviewing the proposed SAMHSA Center or Office response for appropriateness and assisting in finalizing a response;

Establishing a tracking database for complaints, with information on the type of complaint and its disposition, whether the complaint was deemed inapplicable or frivolous, and any resolution or corrective action taken;

Submitting an annual report on behalf of SAMHSA to HHS with the number and types of complaints, and the actions taken, in time for the HHS to report to OMB by January 1 (beginning in 2004);

Posting on the SAMHSA web site any further clarifications, guidelines, and Frequently Asked Questions (FAQs) about handling SAMHSA information complaints;

Making available examples of typical complaints and appropriate responses collected from SAMHSA Center

1. Responsibility of the Complainant

   To seek a correction of information disseminated by the agency, individuals should follow the procedures described below.

   1. A complaint or request for review and correction of information shall be in written hard copy or electronic form;
   2. it shall be sent to the agency by mail or electronic-mail (e-mail); and
   3. it shall state that an information quality request for correction is being submitted.

   The complaint shall contain

   4. a detailed description of the specific material that needs to be corrected including where the material is located, i.e. the publication title, date, and publication number, if any, or the Web site and Web page address (url), or the speech title, presenter, date and place of delivery; and
   5. the specific reasons for believing the information does not comply with OMB, HHS or SAMHSA guidelines and is in error and supporting documentation, if any;
   6. the specific recommendations for correcting the information;
   7. a description of how the person submitting the complaint is affected by the information error; and
   8. the name, mailing address, telephone number, e-mail address, and organizational affiliation, if any, of the individual making the complaint.

   Complainants should be aware that they bear the ‘burden of proof’ with respect to the necessity for correction as well as with respect to the type of correction they seek.

2. Responsibility of the Agency

   Based on a review of the information provided, the agency will determine whether a correction is warranted and, if so, what action to take. The agency will respond to the requestor by letter or e-mail. The agency's response will explain the findings of the review and the actions that the agency will take, if any. The response will consider the nature and timeliness of the information involved and such factors as the significance of the correction on the use of the information and the magnitude of the correction. The response will describe how the complainant may request reconsideration. The agency will respond to all requests for correction within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.

3. Appeals

   If the individual submitting the complaint does not agree with the agency's decision (including the corrective action, if any), the complainant may send a written hard copy or electronic request for reconsideration within 30 days of receipt of the agency's decision. The appeal shall state the reasons why the agency response is insufficient or inadequate. Complainants shall attach a copy of their original request and the agency response to it, clearly mark the appeal with the words, Information Quality Appeal and send the appeal to the specific agency appeals address.

   The agency official who resolved the original complaint will not have responsibility for the appeal.

4. Appropriate Responses

   The agency will respond to all requests for appeals within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.

   SAMHSA may respond in a number of possible ways, including:

   • Issue a written retraction;
   • Suspend further dissemination of the information in question;
   • Reprint the publication with corrections;
   • Refer the complainant to the underlying data;
   • Arrange for an independent re-analysis of the data by SAMHSA or a mutually acceptable third party if the data are not publicly available and the complaint involves "influential scientific or statistical information." Complainants must agree to pay the costs of re-analysis or the process terminates.

VII. Influential Scientific, Financial and Statistical Information

The Information Quality Guidelines require that "influential" scientific, financial, or statistical information in official Government documents must be based on studies that can be substantially reproduced if the original or supporting data were to be independently re-analyzed using the same methods. "Influential" when used in the phrase "influential scientific, financial, or statistical information" means that SAMHSA can reasonably determine that dissemination of the information will have or does have a clear and substantial impact on important public policies or important private sector decisions or will have important consequences for specific health practices, technologies, substances, products, firms, etc.

SAMHSA is committed to applying rigorous scientific standards to ensure the accuracy and reliability of research results. For scientific and technical documents, the scientific community recognizes peer review as the primary mean of quality control.

To facilitate the replication of scientific and other influential information by qualified third parties, SAMHSA continues to encourage the sharing of data and methods where practicable. Since the influence and implications of SAMHSA-disseminated information cannot always be truly anticipated, all SAMHSA scientific reports are expected to state clearly how analytic results were generated -- the specific data and sources used, data collection methodologies, various hypotheses and assumptions, specific analytic methods, statistical procedures, sources of bias or measurement error -- making the analysis sufficiently transparent so as to be capable of being reproduced. SAMHSA advocates the archiving of data through the Substance Abuse and Mental Health Data Archive or similar archive of publicly available data to facilitate the reproducibility of influential information.

VIII. References

HHS Standards of Conduct Regulation (45 CFR 73.735-705). Updated October 1, 2000.

HHS Printing Handbook, September 1998.

Office of Management and Budget Guidelines for Ensuring and Maximizing the Quality, Objectivity, utility, and Integrity of Information disseminated by Federal Agencies. Final Guidelines. January 3, 2002. www.whitehouse.gov/omb/fedreg/reproducible.html

Office of Management and Budget Circular No. A-130 Revised (Transmittal Memorandum No. 4) Management of Federal Information Resources. November 30, 2000.

Endnotes

¹ In general, special terms and conditions of a grant award will specify ownership rights to data. For this purpose, "data" means recorded information, regardless of the form or media on which it may be recorded, and includes writings, films, sound recordings, pictorial reproductions, drawings, designs, or other graphic representations, procedural manuals, forms, diagrams, work flow charts, equipment descriptions, data files, data processing or computer programs (software), statistical records, and other research data. Grantees are required to place an acknowledgment of SAMHSA grant support and a disclaimer, as appropriate, on any publication written or published with such support and, if feasible, on any publication reporting the results of, or describing, a grant-supported activity. An acknowledgment shall be to the effect that: "This publication was made possible by Grant Number ____ from _________" or "The project described was supported by Grant Number ___ from ______" and "Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the (name of awarding component or SAMHSA)."

² The conditions allowing SAMHSA staff to be (co-)authors of publications under SAMHSA extramural awards ordinarily arise only from contracts and cooperative agreements, where, by definition, there is substantial programmatic (i.e., scientific-technical) staff involvement. To merit approval for (co-)authorship on publications from extra-mural awards (including grants, contracts, and other award mechanisms), SAMHSA staff must have played a substantial role beyond normal program officer duties, including the following: originating the specific ideas that led to the research activity and manuscript; performing significant portions of the activity, and participating actively in preparing manuscripts.

1. Office of the Assistant Secretary for Planning and Evaluation

I. Office Mission

The Assistant Secretary for Planning and Evaluation (ASPE) advises the Secretary of the Department of Health and Human Services on policy development in health, disability, human services, and science, and provides advice and analysis on economic policy. ASPE leads special initiatives, coordinates the Department's evaluation, research and demonstration activities, and manages cross-Department activities such as strategic planning, legislative planning and review of regulations. Integral to this role, ASPE conducts research and evaluation studies, develops policy analyses and estimates the cost and benefits of policy alternatives under consideration by the Department or Congress.

II. Scope and Applicability of Guidelines

ASPE will ensure that disseminated information meets the standards of quality set forth in the OMB, HHS and ASPE guidelines.  It is ASPE’s goal to ensure and maximize the quality, objectivity, utility, and integrity of information that it disseminates to the public.  We strive to provide information that is accurate, reliable, clear, complete, unbiased, and useful. We are committed to integrating the principle of information quality into every phase of information development, including creation, collection, maintenance, and dissemination.  The pre-dissemination review described in the guidelines only applies to information disseminated on or after October 1, 2002.  The administrative mechanism for correction applies to information that the agency disseminates on or after October 1, 2002, regardless of when the agency first disseminated the information. The guidelines apply to the following kinds of information that the Office of the Assistant Secretary for Planning and Evaluation (ASPE) disseminates: (1) the reports of research and evaluation projects that ASPE sponsors. These reports are usually the results of research that has been conducted for ASPE by contractors under our direction. (2) The final reports of projects supported by grants from ASPE may also be disseminated by ASPE. (3) ASPE staff occasionally write papers for publication and distribution. These reports are formal documents which have gone through several reviews within both within the author's organization and within ASPE.   The Office also produces materials that are not distributed externally. These materials are intended to inform the policy-making process. These materials consist of briefing papers and technical analyses concerning proposed legislative changes, the requirements of new legislation or descriptions of programs or initiatives. While these papers may eventually be distributed externally after the policy-making process is concluded, as required by the Freedom of Information Act, they are not intended to be disseminated widely and consequently are not covered under these guidelines.  Further, the guidelines do not apply to press releases that support the announcement or give public notice of information that ASPE has disseminated elsewhere.

III. Types of Information Disseminated

• Research and evaluation reports
• Annual plans
• Lists of projects in progress or completed
• Project summaries
• Annotated bibliographies
• Lists of sources of data
• Reports to Congress
• Data files for public use
• Tabulations and charts of data relevant to particular topics, such as welfare dependence and child well-being

IV. Types of Dissemination Methods

The Office of the Assistant Secretary for Planning and Evaluation provides both printed copies and electronic copies of the reports it selects for dissemination. The individual components of ASPE have responsibility for the dissemination of the products of their research and evaluation efforts. Each component maintains mailing lists and conducts direct mailings that are limited to the intended audience of the publication. In addition, some components have developed extensive electronic mailing lists that notify subscribers of the availability of new reports on the ASPE website. In some circumstances, a plan for the dissemination of a specific product may be developed to ensure widespread dissemination to a particular audience. Such a plan might include notices to appropriate media outlets, interviews, and presentations at conferences and other meetings.

V. Agency Quality Assurance Policies, Standards and Processes for Ensuring the Quality of Information Disseminated to the Public

From their inception, research and evaluation projects and any other information developmental efforts  that are undertaken by ASPE are subjected to a quality assurance process. Further, ASPE reviews the quality (including the objectivity, utility, and integrity) of information before it’s disseminated and treats information quality as integral to every step of the development of information, including its creation, collection, maintenance and dissemination. Each project is developed with the knowledge that the purpose, proposed methodology and work plan will have to be defended before a peer review panel. The members of the panel are appointed for at least a year at a time and are technically qualified by their professional training and experience. The process for conducting this peer review has been established for many years and refined as circumstances have changed. Requests for changes in the design of proposed projects are not infrequent. Projects are not approved for funding until the peer review panel is satisfied that the project design and work plan have a reasonable expectation of providing a useful product. All projects involving original large scale data collected from the public undergo an exacting, rigorous multi-level review process in connection with the Paperwork Reduction Act (PRA).  Further, ASPE is committed to demonstrating in its PRA clearance packages that each draft information collection will result in information that will be collected, maintained, and used in a way that is consistent with OMB, HHS and ASPE information quality guidelines.

Reports that are selected for dissemination to the public undergo the following processes: Every report that is received as the result of a grant or contract is reviewed by the project officer and that person’s supervisor. If necessary, a technical expert may be asked to review the report and offer an opinion. A decision is made whether the report may be distributed or not. A further decision is made whether to seek support for a more extensive dissemination effort.

If a more extensive dissemination effort is recommended, a more thorough review process involving other organizations is initiated. This will usually follow formal clearance procedures controlled by the Executive Secretariat in the Office of the Secretary. Offices outside of ASPE, including the Office of the Assistant Secretary for Public Affairs and one or more operating divisions with an interest in the report’s contents are asked to concur in the release of the report and provide comments. Editorial changes may be made as a result of this review.

VI. Agency Administrative Complaint Procedures

ASPE has developed administrative mechanisms to allow affected persons to seek and obtain correction of disseminated information that does not comply with OMB, HHS and ASPE guidelines. Complaints about the quality or accuracy of the information being disseminated in a report or other document should be addressed in writing, by mail to:

Information Quality Act Coordinator
Office of the Assistant Secretary for Planning and Evaluation
200 Independence Ave., S.W.
Washington, D.C. 20201 
or ASPE.InfoQuality@hhs.gov

I. Office Mission

The Office of Public Health and Science (OPHS) provides leadership to the nation on public health and science, and communicates on these subjects to the American people. OPHS is led by the Assistant Secretary for Health (ASH), whose chief interest is promoting, protecting, and improving the nation's health. This role encompasses responsibilities as senior advisor to the Secretary for public health and science and director of program offices housing a variety of essential public health activities. The offices in OPHS are: the Immediate Office of the ASH; the Office of the Surgeon General; the Office of HIV/AIDS Policy; the Office of Population Affairs (OPA); the Office of Disease Prevention and Health Promotion (ODPHP); the President's Council on Physical Fitness and Sports; the Office of Minority Health (OMH); the Office on Women's Health (OWH); the Office for Human Research Protections; the Office of Global Health Affairs (OGHA); the Office of Research Integrity; and the Office of Military Liaison and Veterans Affairs. In addition, the Director of the National Vaccine Program Office and the Regional Health Administrators report to the ASH.

II. Scope and Applicability of Guidelines for Agency/Office

Types of Information Disseminated by the Agency to the Public

The pre-dissemination review described in the guidelines only applies to information disseminated on or after October 1, 2002. The administrative mechanism for correction applies to information that the agency disseminates on or after October 1, 2002, regardless of when the agency first disseminated the information.

The purpose of these Guidelines is to provide guidance to OPHS offices about administrative procedures to ensure the quality of information they disseminate to the public.  The Guidelines also provide guidance to the public about how to file a complaint about the quality of the substantive information disseminated by OPHS offices and how OPHS offices should respond to public complaints.   The Guidelines apply to substantive information disseminated by OPHS offices and representing OPHS/HHS views.   Substantive information includes consumer and professional education materials, scientific and technical reports, policy and program recommendations, research findings from sponsored grants that include a dissemination component, and public speeches representing official HHS policy.   The Guidelines do not apply to information that is labeled with a disclaimer as not representing agency views, intra- or inter- agency information, regulations, compliance oversight reports, grant and program announcements, or information describing basic agency operations.

OPHS offices disseminate a variety of public health and science information to the public. The primary types of information disseminated are consumer and professional education and scientific and technical reports. Information disseminated by OPHS is based on science, derived from state of the art knowledge, and peer-reviewed by experts inside and outside government, depending on the nature of the information.

For example, the Office of Disease Prevention and Health Promotion disseminates information about national disease prevention and health promotion goals and objectives such as the Healthy People 2010 document published in November 2000. The Office on Women's Health disseminates consumer education about women's health issues in the form of pocket planners, annual daybooks, and fact sheets such as the 2002 Women's Health Daybook Living Long, Living Well. The Office of Minority Health disseminates information on minority health issues through two nationally distributed newsletters. The Office of the Surgeon General disseminates information on high priority national public health issues in the form of Surgeon General Reports, Calls to Action and National Strategies such as the Surgeon General's Call to Action to Prevent and Decrease Overweight and Obesity 2001.

III. Types of Dissemination Methods

OPHS offices use both print and electronic methods to disseminate information. Offices use Web sites, clearinghouses (telephone information services), print reports, print brochures and newsletters, fact sheets, and a variety of consumer and professional educational materials, such as bookmarks, wallet cards, day planners, and pocket guides. OPHS staff also makes public speeches representing OPHS/HHS public health policy matters. These speeches are presentations of substantive content, not public affairs events.

IV. Agency Quality Assurance Policies, Standards, and Processes for Ensuring the Quality of Information Disseminated to the Public

It is OPHS policy to ensure and maximize the quality, objectivity, utility, and integrity of information that it disseminates to the public according to the standards set forth in the OMB, HHS and OPHS guidelines. OPHS offices strive to provide information that is accurate, reliable, clear, complete, unbiased, and useful. OPHS offices are committed to integrating the principle of information quality into every phase of information development, including creation, collection, maintenance, and dissemination.

The general standard for information disseminated by OPHS offices is the best available public health and science information. Best available is determined by information published in the highest quality peer-reviewed journals, comparison with best practices as established by the relevant discipline, and reviews by expert panels, individual subject experts, external Advisory Committees, Coordinating Committees with subject experts from HHS agencies, Steering Committees, and staff review.  OPHS reviews the quality (including the objectivity, utility, and integrity) of information before it is disseminated and treats information quality as integral to every step of the development of information, including its creation, collection, maintenance and dissemination. For consumer and professional educational materials, appropriateness of the materials for the intended users is a special focus. Appropriateness is determined through staff review, focus groups, user surveys, audience testing, and dissemination of information and materials for public comment. Depending on the subject matter, disseminated information may also be subject to review by legal staff.

Further, staff strives to collect updated, timely information and remain aware of emerging and newly developed data.  In addition, staff is committed to demonstrating in the Paper Reduction Act (PRA) clearance packages that each draft information collection will result in information that will be collected, maintained, and used in a way that is consistent with OMB, HHS and OPHS information quality guidelines.

V. Agency Administrative Complaint Procedures

OPHS has developed administrative mechanisms to allow affected persons to seek and obtain correction of disseminated information that does not comply with OMB, HHS and OPHS guidelines.

1. Responsibility of the Complainant

   To seek a correction of information disseminated by the agency, individuals should follow the procedures described below. Complainants should be aware that they bear the ‘burden of proof’ with respect to the necessity for correction as well as with respect to the type of correction they seek.

   A complaint or request for review and correction of information must be in written hard copy or electronic form; sent to the agency by mail or electronic-mail (e-mail); and state that an information request for correction is being submitted.

   The complaint shall contain

   1. a detailed description of the specific material that needs to be corrected including where the material is located, i.e. the publication title, date, and publication number, if any, or the Web site and Web page address (url), or the speech title, presenter, date and place of delivery; and
   2. the specific reasons for believing the information does not comply with OMB, HHS or OPHS guidelines and is in error and supporting documentation, if any;
   3. the specific recommendations for correcting the information;
   4. a description of how the person submitting the complaint is affected by the information error; and
   5. the name, mailing address, telephone number, e-mail address, and organizational affiliation, if any, of the individual making the complaint.

   Complaints should be addressed and submitted to:

   Executive Officer
   Office of the Assistant Secretary for Health (OASH)
   U.S. Department of Health and Human Services
   200 Independence Ave, S.W.
   Washington, D.C. 20201

   Alternatively, complaints may be e-mailed to: ASPE.InfoQuality@hhs.gov 

2. Responsibility of the Agency

   Based on a review of the information provided, the relevant OPHS office will determine whether a correction is warranted and, if so, what action to take. The office will respond to the requestor by letter or e-mail. The office's response will explain the findings of the review and the actions that the office will take, if any. The response will consider the nature and timeliness of the information involved and such factors as the significance of the correction on the use of the information, the magnitude of the correction, and the resource requirements for the correction. The response will describe how the complainant may request reconsideration. The office will respond to all requests for correction within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the office will inform the complainant that more time is required and indicate the reason why and an estimated decision date.

3. Appeals

   If the individual submitting the complaint does not agree with the office's decision (including the corrective action, if any), the complainant may send a written hard copy or electronic request for reconsideration within 30 days of receipt of the office's decision. The appeal shall state the reasons why the office response is insufficient or inadequate. Complainants shall attach a copy of their original request and the office response to it, clearly mark the appeal with the words, "Information Quality Appeal" and send the appeal to the OPHS appeals address.

   The office official who resolved the original complaint will not have responsibility for the appeal.  OPHS will respond to all requests for appeals within 60 calendar days of receipt.  If the request requires more than 60 calendar days to resolve, OPHS will inform the complainant that more time is required and indicate the reason why and an estimated decision date.

   Appeals should be addressed and submitted to:

   Executive Officer
   Office of Public Health and Science
   U.S. Department of Health and Human Services
   200 Independence Ave, S.W.
   Washington, D.C. 20201
   ASPE.InfoQuality@hhs.gov

VI. Influential Scientific, Financial and Statistical Information

Given the OPHS mission, from time to time, OPHS offices disseminate information that is regarded as influential. In these instances, OPHS adheres to the highest standards of transparency about information sources, methods and analytical techniques. This influential information is in the form of scientific and technical reports. These reports compile, synthesize, and analyze state-of-the-art knowledge about high priority public health issues that have not previously received sufficient attention. The reports include data from published sources and the public domain, as well as expert opinion, consensus, and recommendations. The sources of underlying data are referenced in reports. Typically, the sponsoring or disseminating office does not conduct original research for these reports, although the office may convene Advisory Groups, Steering Committees, Coordinating Committees or similar bodies as well as take public comment. This input may become part of the report.