Performance Improvement 1998. Evaluation Management


The management of HHS evaluations, which are carried out on a regular basis by the HHS agencies and offices and coordinated by ASPE, involves four basic functions:

1. Evaluation planning and coordination

2. Project management

3. Quality assurance

4. Dissemination of evaluation reports.

A description of each function in general terms follows. Additional information on the individual HHS agencies, ASPE, and OPHS evaluation functions is found in Chapter III.

Evaluation Planning and Coordination

The HHS agencies, ASPE, the Office of the Inspector General (OIG), and OPHS develop evaluation plans annually in concert with HHS's program planning, legislative development, and budgeting cycles. Plan development is coordinated by ASPE. Each agency or office plan generally states the evaluation priorities, or projects under consideration for implementation. Typically, HHS evaluation priorities include congressionally mandated program evaluations, evaluations of Secretarial program or policy initiatives, and assessments of new programs, programs that are candidates for reauthorization, or programs for which key budget decisions are anticipated.

More specifically, HHS evaluation planning activities involve preparing two reports to the Congress. First, those agencies and offices that use the PHS 1-percent evaluation set-aside authority--AHCPR, CDC, HRSA, NIH, ASPE, OPHS, SAMHSA--submit a formal plan to ASPE, which coordinates and develops the individual plans into the HHS report to the Congress on the use of the PHS 1-percent authority. This report must be submitted to the Congress before HHS can implement the plan.

Second, the Congress requests that HHS coordinate all of its research, demonstration, and evaluation programs to ensure that the results of these projects address HHS program goals and objectives. ASPE and the Assistant Secretary for Management and Budget work together with HHS agencies to provide the Congress with a special annual research, demonstration, and evaluation budget plan that coincides with the preparation of the President's fiscal year budget. The plan outlines each HHS agency's research, demonstration, and evaluation priorities as related to its strategic goals and objectives.

Project Management

The design and management of evaluation projects at HHS is principally decentralized--the HHS agencies, OIG, and ASPE all are responsible for executing annual evaluation plans, developing evaluation contracts, and disseminating and applying evaluation results. Even within agencies-- while there is some oversight responsibility and execution capability in the Office of the Director or Administrator--the various subunits (centers, institutes, and bureaus) conduct much of the day-to-day evaluation activity.

The OIG performs independent evaluations through its Office of Evaluations and Inspections (OEI). The OEI's mission is to improve HHS programs by conducting inspections that provide timely, useful, and reliable information and advice to decisionmakers. This information (findings of deficiencies or vulnerabilities and recommendations for corrective action) is usually disseminated through inspection reports issued by the Inspector General. A summary of individual inspection reports and other OIG reports can be viewed on the Internet at OEI also provides technical assistance to HHS agencies in conducting their evaluations.

Quality Assurance

Most evaluation projects are developed at the program level. The initial quality review is generally conducted by a committee of agency- or office-level policy and planning staff members. Before a project is approved, however, it is also reviewed for technical quality by a second committee with expertise in evaluation methodology. Technical review committees follow a set of criteria for quality evaluation practice established by each agency. Some HHS agencies also have external evaluation review committees composed of evaluation researchers and policy experts from universities and research centers. More details on the quality assurance procedures for the various HHS agencies, ASPE, and OPHS are presented in Chapter III.

Dissemination of Evaluation Reports

Maintaining report libraries and distributing information on evaluation results is an important component of HHS evaluation management. Project information and reports are continuously submitted to the HHS Policy Information Center (PIC)--the departmental evaluation data base and library maintained by ASPE. As an information data base and library resource, the PIC contains nearly 7,000 completed and in-progress evaluation and policy research studies conducted by the Department, as well as key studies completed outside of HHS by the U.S. General Accounting Office (GAO) and private foundations.

Typically, the results of HHS evaluations are disseminated through targeted distribution of final reports, articles in refereed journals, and presentations at professional meetings and conferences. Although individual HHS agencies have primary responsibility for disseminating results, there is a departmentwide effort under way to expand dissemination to the larger research and practice communities through centralized computer communications and publications. Abstracts of all studies maintained in the PIC evaluation data base are now accessible on the Internet at Information is available on completed projects, including the name and telephone number of an HHS official responsible for the project.

In addition, HHS widely distributes copies of the annual evaluation report series, beginning with Performance Improvement 1995 and continuing to this report, through the HHS website. For convenience, these reports can be accessed on the Internet at: