This chapter consists of summaries of six evaluation projects selected by an outside review panel as outstanding based on rigor of evaluation methodology, importance of findings, and clarity of presentation.
- The Panel reviewed twenty-six reports nominated by the HHS agencies and selected the reports to be highlighted in this chapter on the basis of the following criteria:Is the report useful? Does it address a significant policy issue or problem? Are the findings likely to be useful for a broader audience?
- Is the report conceptually sound? Is it clearly based upon and integrated with previous research? Are assumptions clearly stated? Is the research appropriately linked to the program? Are relevant stakeholders identified and included?
- Is the report methodologically sound? Are its concepts, designs, data collection, and analyses conducted and reported in a competent manner?
- Are the recommendations practical? Do the conclusions and recommendations logically follow from the data and analyses, and are they relevant to the questions asked?
A full statement of the criteria is found in Appendix D.
Performance measurement or assessment, program management and development, and policy analysis and development represent the three most common uses of HHS evaluation resources. Performance measurement or assessment is a high priority for HHS agencies. The development, implementation, and refinement of programs are more results oriented than in the 1990’s and specific measurements are required under the Government Performance and Results Act (GPRA) of 1993. Program management and development reflects the kind of evaluation projects that program managers initiate to obtain information or data that will help them manage a program more efficiently and ensure successful results. Policy analysis and development includes the evaluation projects conducted by HHS agencies to examine the impact of alternative policies, on the future direction of HHS programs or services.
It is important to note that the context in which an evaluation was conducted may have changed since the evaluation was initiated. It should also be recognized that many evaluations are bounded by policy and resources from the funding agency. In addition, the panel considered whether the timing of the evaluation is appropriate for inclusion – evaluation reports that reflect a mid-program assessment of an ongoing evaluation may be included if the policy it addresses is currently under discussion, if the program is large and recommendations potentially include a need for change in program direction, or for other reasons that indicate a midcourse assessment is appropriate.
Each summary includes a brief abstract; a description of the study, including its purpose, background, methods, findings, and use of results; and the name and phone number of the HHS official to contact for additional information.
Following are the six highlighted reports: