Good evaluation promotes good policy, good programs, and successful outcomes. To achieve this, we need unconstrained, thoughtful, and creative evaluators. Here are some notes from a recent presentation about global evaluation projects that may be relevant in the context of work of the Department of Health and Human Services and that could be helpful to legislative staff framing new or revised authorities under which the Department must act.
These are notes taken at a presentation by Rachel Glennerster, speaking on the subject, Using Scientific Evidence to Fight Poverty: The Role of Randomized Evaluations at an event sponsored by the State Department’s Foreign Affairs Evaluation Coordination Work Group, held in the Truman Building in Washington, DC, September 12, 2008.
- Identify what are the truly important questions.
- Process evaluation is for accountability; impact evaluation seeks a deeper level of accountability.
- Impact evaluation can be done strategically; it doesn’t need to be done everywhere or always.
- Most evaluations are merely review like project officer assessments.
- We need to focus on greater rigor of evidence because we don’t know what is most cost-effective.
- Successful evaluation can yield surprising results (such as, that the most effective way to get children into school in Africa is through wide spread deworming).
- Some ways to design and carry out low cost, high impact, randomized studies include: randomizing around the margin target groups, first finding out what’s already been done, learning from the rigorous impact evaluations of others, and working out the study kinks before conducting a major roll out.
- Identifying key questions to answer makes evaluation work more effective.
- Carry out representative studies, not gold plated versions of the program.
- Do things that you believe can be scaled up if they are found to work.
- Do beta testing of and learn from them.
- Be careful not to evaluate either too early or too late.
- Learn the fundamentals of what makes for good evaluation.
- Ask important questions in order to make sure that evaluation has real value.
- Look for small changes that can have big impacts.