Rapid Evaluation Approaches for Complex Initiatives : Table 2


: Table 2

Evaluation FactorOrganizational ChangeCMS Partnership For Patients Campaign
1.     Situational dynamicsComplicatedComplicated campaign, with some complexity of learning across HENs in an “all teach and all learn” environment.
2.     Intervention complexityComplicated programsNational initiative to reduce preventable patient harms in 3700 hospitals across 27 HENs.
3.     Governance structureFederal funder of multiple grantsCMS is the single federal funder of the PfP.
4.     Scale of outcomesShort list of individual-level outcomesPfP has two overarching outcomes (reduced hospital-acquired patient harms, reported in 11 clinical areas, and hospital readmissions.
5.     Timeline of expected resultsIncremental change expected in monthsResults are expected in months; the focus is on speeding up the pace of change to achieve the goals in three years.
6.     Theory of changeTesting a specific program modelHENs facilitate sharing of best practices among their aligned hospitals and offer hospitals training, technical assistance, learning collaboratives, and reporting systems to help them achieve the PfP goals.
7.     Execution strategyFidelity to work plans outlining program goals, objectives, and strategiesDetailed hospital-specific work plans and measures of success are developed and implemented by the hospitals.
8.     PurposeImplementation and efficacy questionsIn what contexts are the PfP hospitals working to achieve the PfP goals? What progress are the hospitals making? What are early results? How do PfP implementation and early outcomes vary by hospital, HEN, and condition?
9.     Reporting and use of findingsProgram management separates reporting and learning functionsMonthly feedback report to CMS and HENs, and ad hoc reports in response to special requests. No external evaluation linkage to CMS’s internal Learning Team to maintain objectivity.
10.  Rapid evaluation methodsRapid cycle formative and summative evaluation methodsRapid cycle formative and summative evaluation methods; time-series analyses of changes in hospital-specific processes and outcomes, with documentation of hospitals’ contexts, culture, and PfP activities.