This Performance Improvement 2005 report presents to Congress a comprehensive summary of evaluation projects completed by the Department. The Office of the Secretary, and agencies comprising the Department, engage in extensive evaluation activities; some are required by statute, others are determined to be essential by an individual agency, the Department, or the President. Evaluation is a core Federal program management responsibility along with strategic planning, policy and budget development, and program operation (Figure 1, Performance Management System).
Performance Management System
Figure 1
The Department of Health and Human Services (DHHS) is responsible for more than 300 separate programs costing taxpayers over $500 billion annually for health and social service support payments and approximately $70 billion for discretionary programs. DHHS spends about $2.6 billion for research, demonstration, and evaluation activities. Successful evaluations increase the likelihood of effective delivery of public services through these programs and insure that programs are efficient, targeted to their intended clients, and well managed.
-
Structure of this Report
-
Evaluations summarized in this year’s report are organized under the Department’s eight broad Strategic Goals consisting of 40 essential Objectives (Appendix A). As articulated in the Department’s FY 2004-2009 Strategic Plan, the goals are:
I. Prevent Disease and Illness
II. Protect Our Homeland
III. Close the Gaps in Health Care
IV. Improve Health Science
V. Realize the Possibilities of 21st Century Health Care
VI. Work Toward Independence
VII. Leave No Child Behind
VIII. Improve Department Management
Performance Improvement 2005, the 11th annual report in this series, provides summaries of findings of DHHS evaluation projects completed during Fiscal Year 2004 (October 1, 2003 through September 30, 2004). This Chapter provides an overview of the Department’s evolving evaluation responsibilities. Chapter 2 presents the summaries of the evaluations. In addition to the Department’s Strategic Plan Goals and Objectives found in Appendix A, Appendix B contains a statement of each agency’s Mission and Evaluation Program, Appendix C provides a table of the studies by each agency and the Objective in this report where the study may be found, and Appendix D acknowledges the DHHS officials who contributed to the report.
-
-
Role of Evaluation
-
Programs need to: work better for the individuals served, assure that tax dollars are wisely spent, and achieve the aims set for them by elected representatives. This report to Congress on Performance Improvement continues the transition to a more strategic and analytic presentation of evaluation studies. With the implementation of a unified Strategic Plan, as required by the Government Performance and Results Act of 1993, the Department recognizes its responsibility both to evaluate programs and to assure that evaluation funds are targeted to address the core goals and objectives the Congress and Executive branch seek. This re-engineering of management practice underscores the important role of evaluations’ potential to test, weigh, measure and judge the success of management performance, program outputs, and social outcomes.
Evaluations play an integral role in carrying out the DHHS mission. Assessing various aspects of agency functioning allows staff to identify means of improving individual program performance. DHHS evaluations directly support: (1) helping government officials and members of the Congress make decisions related to programs, policies, budgets, and strategic planning; (2) enabling managers improve program operations and performance; and (3) disseminating evaluation results and methodological tools useful to the larger health and human services community of state and local officials, researchers, advocates, and practitioners for improving the performance of their programs.
-
-
Types of Evaluation
-
For DHHS, evaluation is the assessment of the performance (efficiency, effectiveness, and responsiveness) of DHHS programs or strategies through the analysis of data or information collected systematically and ethically, and the effective use of resulting information in strategic planning, program or policy decision making and program improvement. Evaluations serve one or more of the following objectives (Figure 2, Types of Evaluation):
Enhance Program Effectiveness and Support Policy Analysis — Determine the impact of DHHS programs on achieving intended goals and objectives and examine the impact of alternative policies on the future direction of DHHS programs or services.
Improve Performance Measurement — Monitor annual progress in achieving departmental strategic and performance goals. We invest evaluation funds to develop and improve performance measurement systems and improve the quality of the data that support those systems. Performance measurement is a high priority for DHHS agencies. The emphasis during development, implementation, and refinement of programs is on results and specific measurements are required
under the Government Performance and Results Act.Assess Environmental Factors — Seek to understand the forces of change in the health and human services environment that influence the success of our programs and the achievement of our goals and objectives. Such understanding allows us to adjust our strategies and continue to deliver effective health and human services.
Strengthen Program Management and Development — Address the need of program managers to obtain information or data that will help them effectively design and manage programs more efficiently and ensure successful results. Focus on developmental or operational aspects of program activities and provide understanding of services delivered and populations served.
Types of Evaluation
Figure 2
-
-
Evaluation Resources
-
Evaluation activities of the various DHHS agencies are largely supported through two funding mechanisms: direct use of program funds and use of special legislative set-aside authorities for evaluation. The first is a common mechanism by which program managers have discretionary authority to use appropriated program funds to support contracts that will design and implement evaluation studies, and analyze evaluation data. In some cases, a program’s legislative authority calls for a specially mandated evaluation, and program funds are used directly to support the evaluation.
The second mechanism for evaluation funding is the legislative set-aside authority which permits the Secretary of DHHS to use a portion of overall program funds for evaluation purposes. The largest of such set-aside authorities at DHHS is Section 241 of the Public Health Service (PHS) Act for evaluations conducted by several HHS agencies:
Administration for Children and Families (ACF)
Agency for Healthcare Research and Quality (AHRQ)
Centers for Disease Control and Prevention (CDC)
Health Resources and Services Administration (HRSA)
National Institutes of Health (NIH)
Substance Abuse and Mental Health Services Administration (SAMHSA)
In addition, two staff components in the Office of the Secretary receive funds under this authority to carry out evaluations:
Office of the Assistant Secretary for Planning and Evaluation (ASPE)
Office of Public Health and Science (OPHS)
This authority was originally established in 1970, when the Congress amended the Act to permit the DHHS Secretary to use up to 1 percent of appropriated funds to evaluate authorized programs. Section 241 limited the base from which funds could be reserved for evaluations to programs authorized by the PHS Act. Excluded were funds appropriated for the Food and Drug Administration, the Indian Health Service, and certain other programs that were managed by PHS agencies but not authorized by the Act (e.g., HRSA’s Maternal and Child Health Block Grant and CDC’s National Institute for Occupational Safety and Health).1 In addition, programs may have other authorized sources of funding for evaluation activities.2 Section 206 of Division F of the Consolidated Appropriations Act, 2005 increased the amount the Secretary could use for evaluation to 2.4 percent.
1 FDA programs are principally authorized by the Food, Drug and Cosmetic Act and appropriations are provided by the Appropriations Subcommittee on Agriculture, Rural Development, Food and Drug Administration and Related Agencies. IHS programs are principally authorized by the Indian Health Care Improvement Act and the Indian Self-Determination Act; appropriations are provided by the Appropriations Subcommittee on Interior and Related Agencies.
2 The Older Americans Act (OAA) specifies that $1.5 million from Title III, and $1.5 million from Title IV are to be available from its annual appropriations to be used for the evaluation of OAA programs. Since 2000, AoA has used those funds for the Performance Outcome Measures Project and its annual national performance measurement surveys. AoA initiated new evaluation studies of Title III-D Health Promotion and Disease Prevention, and Title III-B Supportive Services in FY 2004 and intends to continue evaluating all OAA titles on a rotating basis in the future.
-
-
Evaluation Management
-
Management of evaluations carried out by DHHS agencies and offices and coordinated by ASPE involves:
(1) planning and coordination, (2) project oversight, (3) quality assurance, and (4) dissemination of results (Figure 3, Evaluation Management Cycle). A description of each function follows.
Evaluation Management Cycle
Figure 3
-
View full report
"pi_2005.pdf" (pdf, 750.85Kb)
Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®