The development of data and information within HHS is generally undertaken within the context of two overarching goals:
- attention to information quality as a total and continuing process, and
- commitment to making data and information supported with public funds available to the public, consistent with confidentiality concerns and resource availability.
Further, when HHS agencies prepare a Paperwork Reduction Act (PRA) clearance submission, they strive to engage in a data development effort that will result in information that will be collected maintained, and used in a way that is consistent with OMB, HHS, and agency-specific information quality guidelines. As a general policy, HHS views data and information quality as a continuing process that begins at the inception of the information development process with project conceptualization and carries through all phases of data planning, design and execution, including information dissemination activities. Further, HHS agencies that support or sponsor research and statistical activities are encouraged to not only describe the methods and data sources in a clear and transparent manner in the reports they release, but also to make the data used for the report available to the public through public use data files, restricted access research files, research data centers, data archives and other mechanisms consistent with confidentiality, legal and proprietary restrictions.
In addition, HHS agencies review the quality (including the objectivity, utility, and integrity) of information before it is disseminated and treat information quality as integral to every step of the development of information, including its creation, collection, maintenance and dissemination. The quality assurance processes that are used to ensure the quality of specific categories of information are described below.
- Research and scientific studies disseminated by HHS are subject to an external, objective peer review process at both the inception stage and the pre-dissemination stage as a part of the publication process in peer reviewed journals. In addition, the quality of all of the intramural research programs of HHS agencies is continually reviewed and monitored by advisory committees and boards of scientific advisors. In accordance with widely accepted scientific research practice in the U.S., research reports disseminated by HHS agencies describe the methods, data sources, analytical techniques, measures, assumptions and limitations of the research, so that the study could be substantially reproduced. If original data are employed, it is the policy of HHS to make every effort to make the data available to the public in de-identified form consistent with confidentiality requirements, proprietary restrictions and resource availability.
- Statistical activities of HHS agencies are based on reliable data sources and are carried out in accordance with modern statistical theory and practice, including scientific sampling, statistical inference and analytical techniques and practices. All statistical programs employ or have access to experts in statistics and research design. HHS houses the National Center for Health Statistics, the federal government's designated general purpose statistical agency for health statistics, as well as programmatic and special purpose statistical activities. All proposals for original data collection activities in HHS undergo a rigorous and exacting review process in connection with the Paperwork Reduction Act, which also provides opportunity for public comment in the design of the information collection. Frequent meetings with user groups are common, and individual surveys and statistical systems often employ project specific technical advisory groups. Substantive reports from HHS statistical activities undergo a quality review process within their organizations before they are released, including expert review by supervisors, internal peer review by qualified scientists and statisticians, and in some cases external peer review as well as expert review by other offices prior to dissemination.
- Programmatic and administrative information — A significant amount of substantive information is disseminated by HHS agencies in connection with and as a byproduct of the administration of programs. Often the programmatic and administrative information disseminated is obtained from third parties, such as States, grantees, health plan contractors or intermediaries, or community-based organizations. In their stewardship function, agencies often collect, compile, standardize, analyze and disseminate such programmatic information. While the reliance on third parties places limits on the federal quality assurance authorities, a variety of techniques are employed to promote the accuracy, completeness and timeliness of the information. These include use of generally accepted accounting and financial management procedures and principles, internal controls, legal certifications and assurances on the part of the organizations supplying the information, audited financial reports and statements, as well as sample audits and site visits, and checks for completeness and consistency with trends and external controls. Programmatic reports are typically subject to supervisory review before release.
- Program evaluation studies are often undertaken by HHS agencies to assess program functioning and identify opportunities for improvement. Agencies employ quality assurance procedures in the choice and development of evaluation projects. Proposals for evaluation activities in HHS agencies are usually reviewed by agency management for suitability, utility and methodology in accordance with sound evaluation design and standards of evaluation practice. Many evaluation projects have specific technical advisory committees that oversee the design and conduct of the evaluation in accordance with standard evaluation theory and practice, and they often provide an expert review of the final report. Results of evaluation activities are released to the public only after agency management has completed a review of the quality, accuracy and completeness of the report.
- Regulatory information — A variety of information is used in support of regulatory development and decision-making. Regulatory activity undertaken by HHS agencies closely tracks statutory authorities and program responsibilities. Scientific, financial, and statistical information used in support of regulatory decision-making is subject to a quality review process within the agency involving appropriate experts depending upon the nature of the information. In addition, all significant proposed regulations are reviewed by OMB prior to issuance for public comment, and all proposed regulatory actions provide for extensive public comment.
- Authoritative health, scientific and consumer information — Several HHS agencies develop and disseminate authoritative health and human services information intended for consumers and the professional community. In some instances, the agency simply provides a link to information developed by other authoritative organizations. In other instances, the agency develops its own consumer and professional practice information. In the latter case, the information is reviewed for scientific and medical accuracy and completeness by experts within the agency before it is disseminated. In a number of instances the information also is reviewed by scientific and medical advisory bodies before dissemination as well, depending on the nature of the information.
- Public health surveillance and epidemiological information — Several HHS agencies compile, analyze and disseminate information from public health surveillance systems and epidemiological activities. In many surveillance systems, the primary information is developed by State and local government agencies, clinical laboratories and other health care entities and reported to CDC for national aggregation and analysis. Data quality is assured through use of reliable data sources, appropriate statistical techniques, agreement on national reporting standards, quality control procedures, standard case definitions and reports, adherence to professional practices and standards for public health reporting in the U.S., and frequent consultation with the user community. Before such information is disseminated, it is reviewed for medical, scientific and public health accuracy, soundness and utility by agency experts. Comment and feedback is encouraged on such information, and HHS agencies work closely with the relevant professional and public health organizations.