HHS Office of Inspector General
- Office Mission
The mission of the Office of Inspector General, as mandated by Public Law 95-452 (as amended), is to protect the integrity of Department of Health and Human Services programs, as well as the health and welfare of the beneficiaries of those programs. The OIG has a responsibility to report both to the Secretary and to the Congress program and management problems and recommendations to correct them. The OIG's duties are carried out through a nationwide network of audits, investigations, inspections and other mission-related functions performed by the OIG components
- Scope and Applicability of Guidelines for Office
OIG will ensure that disseminated information meets the standards of quality set forth in the OMB, HHS and OIG guidelines. It is OIG?s policy to ensure and maximize the quality, objectivity, utility, and integrity of information that it disseminates to the public. We strive to provide information that is accurate, reliable, clear, complete, unbiased, and useful. We are committed to integrating the principle of information quality into every phase of information development, including creation, collection, maintenance, and dissemination.
The OMB Information Quality Guidelines require OIG to evaluate and identify the types of OIG information that will be subject to the Guidelines. The pre-dissemination reviewed described in the guidelines only applies to information disseminated on or after October 1, 2002. The administrative mechanism for correction applies to information that the agency disseminates on or after October 1, 2002 regardless of when the agency first disseminated the information. This section identifies the types of information covered, and also lists the types of information that are exempt.
- Covered Information
OIG reports prepared for use by Department officials formulating broad program policies which are not subject to review by the affected individual(s) or entity prior to public dissemination.
Editorials or Open Letters to the public representing position or view of the agency .
- Information Not Covered
Documents that are summary in nature and where the source documents are already covered under the Guidelines, this would include the Semiannual Report, the Orange Book and the Red Book.
Planning documents that represent future work which has not been completed. This includes the annual OIG Work Plan.
Findings and determinations or agreements made in the course of adjudication. This includes the List of Excluded Individuals and Entities and Corporate Integrity Agreements. Affected parties are subject to and/or can contest exclusion decisions through an adjudicatory process.
Proprietary information owned by another agency, which the Department does not have authority to release to outside sources, such as the Annual Report of the State Medicaid Fraud Control Units.
Documents where the subject individual or entity is already afforded an opportunity to comment on the accuracy of the information to ensure a fair, objective and complete report, this includes most Audit reports.
Advisory Opinions, which are requested of the OIG and are binding only on the requestor.
The guidelines do not apply to press releases that support the announcement or give public notice of information that OIG has disseminated elsewhere.
Documents which comply with the Federal Register standards for public notices. This includes Fraud Alerts, Special Advisory Bulletins and Safe Harbor Regulations, which are published in the Federal Register. As such, the public is accorded the administrative procedures for public notices and afforded the opportunity to comment. The safe harbors also are excluded because they serve as OIG policy.
Guidance issued by the OIG where compliance is voluntary such as those issued under the Compliance Guidance Program. The compliance guidance is based on the Centers for Medicare and Medicaid Services (CMS) regulations and guidelines, any requirements imposed in corporate integrity agreements negotiated by the OIG and input from the affected industry, thus there is an established quality control process through input from the affected entities.
Information or documents explicitly excluded by the Quality Control Guidelines such as press releases, public filings, subpoenas, or adjudicative processes.
Procedural and policy manuals that are produced primarily for internal use.
- Types of Information Disseminated
The types of information published by the OIG is described in Section II.
- Types of Dissemination Methods
Hard copy reports and electronic media such as the Internet. Interested persons can subscribe to receive information electronically on recently issued reports, press releases and other documents.
- Agency Quality Assurance Policies, Standards and Procedures for Ensuring the Quality of Information Dissemination to the Public
The quality assurance process begins at the inception of the information development process.
Information released by OIG is developed from reliable data sources utilizing accepted methods for data collection and is based on thoroughly reviewed analyses and models. The guidelines below describe procedures that OIG employs to assure the quality of its information products. Quality is an encompassing term comprising utility, objectivity, integrity, and reproducibility.
Utility involves the usefulness of the information to its intended audience. The purpose of OIG documents and information is to identify fraud, waste and abuse and to protect the integrity of HHS programs. The OIG issues findings and recommendations on the efficiency, effectiveness, and vulnerabilities of departmental programs.
Objectivity involves a focus on ensuring that information is accurate, reliable and unbiased and that information products are presented in an accurate, clear, complete and unbiased manner. Objectivity is achieved by using reliable data sources and sound analytical techniques, and preparing information products that use proven methods by qualified individuals that are carefully reviewed. Below is a description of the quality assurance guidelines that are used to ensure objectivity and accuracy of information.
OIG reviews the quality (including the objectivity, utility, and integrity) of information before its is disseminated and treats information quality as integral to every step of the development of information, including its creation, collection, maintenance and dissemination.
Quality Assurance Process
The quality assurance procedures is a collaborative, team effort by staff who conduct program inspections and staff who assist, guide, and review written and oral inspection plans and products. Any individual involved with a particular inspection is responsible for helping assure quality of work done and products released. Further, OIG is committed to demonstrating its Paper Reduction Act (PRA) clearance packages that each draft information collection will result in information that will be collected, maintained, and used in a way that is consistent with OMB, HHS, and OIG information quality guidelines.
Specific responsibility for conducting and ensuring quality program inspections rests with Regional Inspectors Generals (RIGs). RIGs assign a Team Leader and other team members to each inspection. This team is responsible for doing quality inspection field work -- including design, data collection, data analysis, and written and oral reports. RIGs must certify for final reports that the inspection was done in accordance with OIG procedures and PCIE quality standards.
If an audit methodology is used to develop the product, the Government Auditing Standards are employed as described below:
- Government Auditing Standards State that "Each audit organization conducting audits in accordance with these standards should have an appropriate internal quality control system in place and undergo an external quality control review."
- This control system is to provide reasonable assurance that the audit organization (1) has adopted, and is following, applicable auditing standards (which includes reporting "the views of responsible officials") and (2) has established, and is following, adequate policies and procedures.
- To determine whether in fact the audit organization's quality control system is operating effectively, the organization is required by standards to have "an external quality control review at least once every 3 years by an organization not affiliated with the organization being reviewed."
OIG's quality control program ensures that audit work performed meets government auditing standards. It consists of two elements:
- The OIG System of Quality Control consists of independent report referencing, Headquarter Desk Reviews and Internal Quality Control Reviews. The latter includes review of working papers related to selected reports.
- External Reviews -- These reviews, also known as peer reviews, are performed by Federal auditors outside the OAS. The external review is required by the Comptroller General's standard on quality control and should be conducted at least once every three years.
Report Validation is a process that is used to help ensure that the facts, findings, conclusions, and recommendations contained in OIG reports are accurate, reliable, and supportable by inspection work papers and analysis.
Report validation is conducted prior to submitting the draft report for IG signature. For objectivity, validation is performed by regional staff that are not a part of the inspection team. To facilitate this process, the draft report is cross indexed to applicable work papers. Work papers must include documentation that inspection teams verified data presented in draft reports by checking it against supporting evidence in the work papers.
To help assure quality and impact of its inspection results, OIG obtains comments and other input from applicable agency staffs on inspection plans, results, and selected products. Inspection teams obtain such input in a variety of ways, including entrance, exit, and other conferences; and comments on inspection designs, data collection instruments, and draft and final reports. Inspection teams use such meetings and reviews to help identify client needs, identify relevant data for accomplishing inspection purposes, verify accuracy and reliability of data collected, and verify soundness of findings, conclusions, and recommendations developed.
Reliability of Data Sources
To maintain credibility, OIG must take reasonable steps to assess the reliability of pre-existing computerized data used as the basis for inspection findings and recommendations. Many inspections either begin with a computerized sample selection or are based entirely on analyzing data extracted from computerized records not under OIG?s direct control. Project staff do not assume that such computer extracts or sample selections are complete or that they accurately reflect the universe of people or transactions being studied.
Data reliability means the degree to which data extracted from computer records for a program inspection completely and accurately reflect the individuals or transactions being studied. This is a relative concept, one that recognizes that data with errors may still be usable, if the errors are not of a magnitude that would cause a reasonable person to doubt findings or conclusions that are based on the data.
To provide reasonable assurance of computerized data reliability we:
- Identify prior reviews by OIG, GAO or by system managers attesting to the computer system and data reliability,
- Review the data dictionary, if it exists, for the database to assure a full understanding of the relevant data elements? structure, content, how the elements are derived, and their interrelationships before requesting data extractions,
- Obtain frequency counts of critical data elements to determine if the data selection criteria are providing the information anticipated,
- Conduct data accuracy tests to ensure that required data elements have been provided and are in the expected format,
- Obtain detailed printouts for a sub-sample of records included in the data extract to confirm that the extraction produced the types of records sought and the required information from those records,
- Obtain source documents (e.g., claim folders) for a sample of extracted records to determine the validity of the data contained in the automated records, if the data reliability is questionable, and include a statement in the Methodology section of draft and final reports describing the extent of reliability testing performed and our confidence in the data used.
Integrity refers to the security of information from unauthorized access or revision to ensure that the information is not compromised through corruption or falsification. To ensure the integrity of information, OIG has in place rigorous controls that have been identified as representing sound security practices.
OIG is highly protective of the confidentiality of information it holds through its policies and practices. OIG has in place programs and policies for securing OIG resources as required by the Government Information Security Reform Act (P.L. 106-398, Title X, Subtitle G). OIG is subject to statutory requirements to protect the sensitive information it gathers and maintains on individuals.
If an agency is responsible for disseminating "influential" information, guidelines for dissemination should include a high degree of transparency about data and methods to facilitate its reproducibility by qualified third parties. Information is considered influential if it will have a substantial impact on important public policies or important private sector decisions. Since many of the OIG's Inspection reports have an impact on important public policies, OIG's information that is subject to section 515 should be highly transparent and capable of being reproduced by qualified persons.
OIG's guidelines call for identification and documentation of data sets used in producing estimates and projections and clear description of the methodology used to produce the analytical results. Some results included in OIG reports are not directly reproducible by the public because the underlying data sets used to produce them are confidential. However, those inspections that are based on publically available data and are made available on request are fully reproducible by the public.
Agency Administrative Complaint Procedure
OIG has developed administrative mechanisms to allow affected persons to seek and obtain correction of disseminated information that does not comply with OMB, HHS and OIG guidelines.
Requests for review should to be submitted in writing to the OIG at the following address:
Public Affairs Officer
Office of Inspector General
Office of Management and Policy
Room 5541 Cohen Building
Washington D.C. 20201
Alternatively, they can be e-mailed to: DataQuality@hhs.gov
- Responsibility of the Complainant
To seek a correction of information disseminated by the agency, individuals should follow the procedures described below.
- A complaint or request for review and correction of information shall be in written hard copy or electronic form;
- it shall be sent to the agency by mail or electronic-mail(e-mail); and
- it shall state that an information quality request for correction is being submitted.
The complaint shall contain
- a detailed description of the specific material that needs to be corrected including where the material is located, i.e. the publication title, date, and publication number, if any, or the website and web page address (url), or the speech title, presenter, date and place of delivery; and
- the specific reasons for believing the information does not comply with OMB, HHS or OIG guidelines and is in error and supporting documentation, if any;
- the specific recommendations for correcting the information;
- a description of how the person submitting the complaint is affected by the information error; and
- the name, mailing address, telephone number, e-mail address, and organizational affiliation, if any, of the individual making the complaint.
- Complainants should be aware that they bear the ?burden of proof? with respect to the necessity for correction as well as with respect to the type of correction they seek.
- Responsibility of the Agency
Based on a review of the information provided, the agency will determine whether a correction is warranted and if, so what action to take. The agency will respond to the requestor by letter or e-mail. The agency?s response will explain the findings of the review and the actions that the agency will take, if any. The response will consider the nature and timeliness of the information involved and such factors as the significance of the correction on the use of the information and the magnitude of the correction. The response will describe how the complainant may request reconsideration. The agency will respond to all requests for correction within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.
If the individual submitting the complaint does not agree with the agency's decision (including the corrective action, if any), the complainant may send a written hard copy or electronic request for reconsideration within 30 days of receipt of the agency's decision. The appeal shall state the reasons why the agency response is insufficient or inadequate. Complainants shall attach a copy of their original request and the agency response to it, clearly mark the appeal and the envelope with the words, "Information Quality Appeal," and send the appeal to:
Public Affairs Officer
Office of Inspector General
Office of Management and Policy
Room 5541 Cohen Building
Washington D.C. 20201
The agency official who resolved the original complaint will not have responsibility for the appeal. The agency will respond to all requests for appeals within 60 working days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.
Influential scientific, financial and statistical information
Addressed in Section V.
Special Considerations for Dissemination
All OIG inspections are conducted in accordance with the Quality Standards for Inspections issued by the President's Council on Integrity and Efficiency. The following is an outline of these standards.
Qualifications: Individuals assigned to perform inspection work must collectively possess adequate professional proficiency for the task required.
Independence: Individuals performing inspection work must be free from impairments that hinder objectivity. Inspectors must consistently maintain an independent, objective attitude and appearance, and shall be subject to supervisory guidance and review to preclude actual or perceived impairments or bias in conducting inspection work and presenting results.
Due Professional Care: Due professional care will be used in conducting inspection work and in preparing reports of other products.
Quality Control: To ensure quality and to expedite the progress of an inspection, proper supervision will be exercised from the start of an inspection to completion of the final inspection report.
Planning: To ensure adequate planning, inspection work will be coordinated, researched, and designed to achieve the objectives of the inspection.
Data Collection: Information and data obtained about the organization, program analysis activity, or function being inspected should be consistent with inspection objectives and sufficient enough to provide a reasonable basis for reaching conclusions.
Evidence: Evidence supporting inspection conclusions should be competent and relevant and lead a prudent person to the same conclusion as that of the inspectors.
Supporting: All relevant information generated, obtained, and used in Documentation supporting inspections findings, conclusions, and recommendations should be retained.
Timeliness: inspectors should seek to deliver significant information to appropriate management officials in a timely manner.
Fraud and Other: If during or in connection with an inspection, inspectors become Illegal Acts aware of illegal acts, or indications of such acts, they should promptly present such information to their supervisors for review and possible referral to the appropriate investigative office.
Reporting: All inspection reports shall present factual data accurately, fairly, and objectively, and present findings and conclusions in a persuasive manner.
Follow-up: Appropriate follow-up will be performed to assure that any recommendations made to agency officials are adequately considered and appropriately addressed.
All OIG audits are conducted in accordance with Government Auditing Standards and OAS policy. Both (1) afford affected entities the opportunity for corrections and (2) require a stringent quality control program.
- Auditee Response
- Government Auditing Standards
Government Auditing Standards state that "Auditors should report the views of responsible officials of the audited program concerning auditor's findings, conclusions, and recommendations, as well as corrections planned."
- Obtaining comments is one of the most effective ways to ensure that a report is fair, complete, and objective.
- Advance comments should be objectively evaluated and recognized, as appropriate in the report.
OIG Audit Policy
OIG Audit policy requires that reports give recognition to the views of the auditee.
- The auditee's formal response to each finding should be included in the final audit report.
- The official position of the auditee should be in writing and should be signed by the responsible official.
- The auditee may present new information in formal written comments to the draft report. In these instances, the information should be evaluated prior to incorporating the comments and issuing the final report.
- The auditee comments would appear in the final report in three places, report summary, individual finding and appendix to report.
- The OIG regards the absence of a response to audit findings and recommendations as a departure from generally accepted government auditing standards. If the auditee does not provide comments after receiving the draft report, this is stated in the final audit report.
- OIG Audit policy requires that respond to each relevant auditee comment.
Quality Control Program
- Government Auditing Standards
Government Auditing Standards State that "Each audit organization conducting audits in accordance with these standards should have an appropriate internal quality control system in place and undergo an external quality control review."
This control system is to provide reasonable assurance that the audit organization (1) has adopted, and is following, applicable auditing standards (which includes reporting "the views of responsible officials") and (2) has established, and is following, adequate policies and procedures.
To determine whether in fact the audit organization's quality control system is operating effectively, the organization is required by standards to have "an external quality control review at least once every 3 years by an organization not affiliated with the organization being reviewed.
- OIG Audit Policy
OIG's Audit quality control program ensures that work performed meets government auditing standards. It consists of two elements:
- A System of Quality Control which consists of independent report referencing, Headquarter Desk Reviews and Internal Quality Control Reviews. The latter includes review of working papers related to selected reports.
- External Reviews -- These reviews, also known as peer reviews, are performed by Federal auditors outside the OIG. The external review is required by the Comptroller General?s standard on quality control and should be conducted at least once every three years.
Other Agency Specific Policies and Procedures
Office of Evaluation Procedures Manual, Data Analysis -- oig.hhs.gov/organization/OEI/other/doc_m.pdf (currently under revision)
Quality Standards for Federal Offices of Inspector General -- www.ignet.gov/pande/standards/igstds.pdf
Quality Standards for Inspections -- oig.hhs.gov/organization/OEI/other/qsidoc.pdf
Government Auditing Standards
Quality Standards for Federal Offices of Inspector General
OMB Circular A-123 (Revised June 21, 1995)