Rapid Evaluation Approaches for Complex Initiatives. Situational Fit


When is it appropriate to use each type of rapid evaluation method? The right design is one that best fits the evaluation’s purpose(s) and captures the complexities of the intervention and its environment (Funnell and Rogers 2011). When system dynamics are not considered in an evaluation’s design, the evaluation will inevitably miss crucial aspects of the intervention and its environment that are affecting the intervention’s implementation, operation, and results. Factors to consider in selecting rapid evaluation methods include the dynamics of the intervention’s context; the structure, program logic, and intended outcomes of the intervention itself; and the intended purpose and use of intervention’s evaluation (Hargreaves 2010). To aid the matching process, these evaluation factors and methods are organized by level of complexity in a comparative framework (Table 1). More details of the evaluation factors are presented next.

Table 1. Rapid Evaluation Dynamics Comparative Framework

Evaluation Element Evaluation Factor Process Change Organizational Change Systems Change
Context 1.  Situational dynamics Simple Complicated Complex
2.  Type of intervention Simple projects Complicated programs Complex initiatives
3.  Governance structure Single organization Federal funder of multiple grants Alliance of multiple funders and stakeholders
Intervention 4.  Scale of outcomes Single, discrete process changes Short list of individual-level outcomes Large-scale population or system-wide change
5.  Timeline of expected results Immediate change expected within weeks Incremental change expected in months Transformative change expected in months or years
6.  Theory of change Implementing an evidence-based practice Testing a specific program model Applying change principles to strategic leverage points
7.  Execution strategy Fidelity to a set of documented procedures Fidelity to work plans outlining program goals, objectives, and strategies Change strategies are developed and revised as the initiative evolves
8.  Purpose Implementation, efficacy, and outcome questions Implementation, efficacy, and outcome questions Implementation and efficacy questions
Evaluation 9.  Reporting and use of findings Unit operations managers and staff receive and use evaluation results Program management separates internal and external reporting and learning functions Strategic leadership incorporates findings into adaptive management cycle
10.   Rapid evaluation methods Quality improvement—plan-do-study-act cycle Rapid cycle evaluation—formative and summative evaluation Developmental evaluation, systems change evaluation, action research methods


View full report


"rs_EvalApproach.pdf" (pdf, 598.02Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®