Skip to main content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Willing, Able -> Ready: Basics and Policy Implications of Readiness as a Key Component for Scaling up Implementation of Evidence-Based Interventions

Publication Date
Aug 31, 2014

DEPARTMENT OF HEALTH & HUMAN SERVICES

Office of the Secretary

Washington, DC
 
OFFICIAL BUSINESS
Penalty for Private Use $300

This brief is one in a series exploring issues related to the implementation of evidence-based interventions. It describes the basics of readiness using the R=MC2 (Readiness = Motivation × General Capacity and Intervention-Specific Capacity) heuristic, examines some of the policy implications of readiness, and identifies directions for future research. “Readiness” refers to the extent to which an organization is both willing and able to implement a particular practice. An emerging body of scholarly work identifies three components of readiness that organizations should address when implementing new EBIs: (a) motivation of people within the organization to adopt new EBIs, (b) general organizational capacities, and (c) intervention-specific capacities. Motivation includes beliefs about an intervention and support for the program which contribute to the desire to adopt a practice. General capacity speaks to different aspects of organizational functioning such as culture, climate, staff capacity, and leadership. Intervention-specific capacity describes human, technical, and fiscal conditions such as knowledge, skills, and intervention-related abilities that are important to the successful implementation of a particular practice.

"

About This Issue Brief

This issue brief was written by Allison Dymnicki, Ph.D., of the American Institutes for Research, Abraham Wandersman, Ph.D., of the University of South Carolina, David Osher, Ph.D., of the American Institutes for Research, Violanda Grigorescu, M.D., of the Centers for Disease Control and Prevention, and Larke Huang, Ph.D.,of the Substance Abuse and Mental Health Services Administration

In 2012, ASPE awarded the American Institutes for Research to manage the Investing in What Works (IWW) project, to continue ASPE’s efforts to keep building the knowledge and supports that evidence-based programs and initiatives can use to improve the quality and
outcomes of interventions funded through federal investments.
 
Office of the Assistant Secretary for Planning and Evaluation
 
Office of Human Services Policy
 
U.S. Department of Health and Human Services
 
Washington, DC 20201

 

Executive Summary

As more federal agencies recommend that organizations implement evidence-based interventions (EBIs), it is critical to know whether an organization is ready to implement these approaches. “Readiness” refers to the extent to which an organization is both willing and able to implement a particular practice. An emerging body of scholarly work identifies three components of readiness that organizations should address when implementing new EBIs: (a) motivation of people within the organization to adopt new EBIs, (b) general organizational capacities, and (c) intervention-specific capacities. Motivation includes beliefs about an intervention and support for the program which contribute to the desire to adopt a practice. General capacity speaks to different aspects of organizational functioning such as culture, climate, staff capacity, and leadership. Intervention-specific capacity describes human, technical, and fiscal conditions such as knowledge, skills, and intervention-related abilities that are important to the successful implementation of a particular practice.

Although a scientific body of knowledge about readiness already exists, there is a need to develop more concrete recommendations for federal agencies and practitioners to use when implementing EBIs. This brief establishes the basics of readiness using the R=MC2 (Readiness = Motivation × General Capacity and Intervention-Specific Capacity) heuristic, examines some of the policy implications of readiness, and identifies directions for future research.

Key Take Away-Messages

  • When considering an organization’s readiness for implementing EBIs, it is helpful toconsider the constructs of motivation, general capacity, and intervention-specificcapacity. This relationship is represented in the heuristic R=MC2.
  • Policymakers may include targeted questions about readiness in funding opportunityannouncements (FOAs) and develop criteria to evaluate answers to these questions toincorporate information about readiness when choosing grantees.
  • Practitioner organizations (e.g., community-based organizations, schools) should assesstheir readiness before implementing an EBI to better understand their technical assistanceneeds. They should also assess organizational readiness throughout the life cycle of aprogram to foster continuous program improvement.
  • Policymakers interested in future research on readiness could consider determiningcommonalities and differences in FOAs across federal agencies, and researching thedifferential effectiveness of different types of technical assistance provided based on thereadiness of an organization.
  • Researchers should explore the weights of various subcomponents of the R=MCheuristic (i.e., the relative importance of different aspects of motivation) to establish highpriority areas for training and technical assistance (TTA).

Introduction

Program administrators and staff are increasingly called to use evidence-based interventions (EBIs) and practices1 to improve outcomes for children and youth across a variety of areas. However, an often unmeasured but necessary precondition for undertaking program improvement activities is that an organization be “ready” for the change; that is, staff are motivated and capable of making the adjustments required. Readiness (or lack thereof) is key to whether a change will take hold successfully or fail to catch on. Readiness to implement EBIs effectively influences whether the time, energy, and money dedicated to new programs will be well spent. 

The purpose of this brief is to describe the role of readiness in implementing and scaling up EBIs. Issues that are related to implementation readiness and EBI scale up are relevant to producing knowledge about what works, and for moving science into practice in socially significant ways. Readiness as conceptualized here, involves the extent to which the members of an organization as well as the organization as a whole are motivated, individually as well as strategically, and have the capacity to implement an intervention with quality. This brief is targeted primarily to staff working in federal agencies but it is also relevant to federal and state policymakers, practitioners, researchers, evaluators at all levels, and consumers—those who will ultimately benefit from the successful implementation of EBIs at scale.


1 We define “evidence-based interventions” as programs, practices, or policies that have been proven to positively change the problem being targeted by a body of scientific knowledge, usually including some form of evaluation (Kratochwill & Shernoff, 2003).

What Are the Core Components of Readiness?

This brief builds on the readiness literature and in particular a conceptualization of readiness proposed by Scaccia and colleagues (2014). Three components of readiness should be considered when implementing an EBI that is new to an organization: motivation, general capacity, and intervention-specific capacity.2 The three components can be depicted as R=MC2 (Readiness = Motivation × General Capacity × Intervention-Specific Capacity).


2 The focus of this brief is on the readiness of an organization and/or the community where it is located. Equally important, but beyond the scope of this brief, is the readiness of an EBI to be scaled up and the quality of evidence that suggests that a practice is “evidence-based.”

To What Extent Is the Organization Motivated To Implement the Intervention?

Scaccia et al. (2014) define motivation as being influenced by the “perceived incentives and disincentives that contribute to the desirability” to use an intervention. We see motivation as both individual and organizational. Motivation is specific to the intervention being implemented. It includes beliefs about and support for the intervention—such as collective expectations, attributes of an intervention, anticipated outcomes of an intervention, pressures for change, and emotional responses. Table 1 presents a list of the subcomponents of motivation3.

Table 1. Subcomponents of Motivation

Subcomponents of MotivationDefinition of Subcomponent
Relative AdvantageDegree to which a particular intervention is perceived as being better than what it is being compared against; can include perceptions of anticipated outcomes.
CompatibilityDegree to which an intervention is perceived as being consistent with existing values, cultural norms, experiences, and the needs of potential users.
DoabilityDegree to which intervention is perceived as relatively difficult to understand and use.
TrialabilityDegree to which an intervention can be tested in a pilot fashion before going to scale.
ObservabilityDegree to which outcomes that result from the intervention are visible to others.
PriorityExtent to which the intervention is regarded as more important than other interventions.

Source: Scaccia et al. (2014)


3 For each of the three readiness constructs, the key subcomponents identified from a systematic review of the readiness literature are presented. This is not an exhaustive list of all subcomponents that could comprise each readiness construct.

What Is the General Capacity of the Organization That Will Implement the Intervention?

General organizational capacities are related to maintaining a functioning organization (e.g., sufficient staffing, effective organizational leadership) and connecting with other organizations and the community. This includes the context, culture, current infrastructure, and organizational processes of the organization where the EBI will be implemented. General capacities are associated with the ability to implement any intervention (Flaspohler, Duffy, Wandersman, Stillman, & Maras, 2008) and include human, fiscal, technical, and evaluative categories. Table 2 presents a list of general capacities.

Table 2. Subcomponents of General Capacities

Subcomponents of General CapacitiesDefinition of Subcomponent
CultureExpectations about how things are done in an organization; how it functions.
ClimateHow employees collectively perceive, appraise, and feel about their current working environment.
Organizational InnovativenessGeneral receptiveness toward change; i.e., an organizational learning environment.
Resource UtilizationHow discretionary/uncommitted resources are devoted to interventions.
LeadershipWhether power authorities articulate and support organizational activities.
StructureProcesses that influence how well an organization functions on a day-to-day basis.
Staff CapacityGeneral skills, education, and expertise that staff possess.

Source: Scaccia et al. (2014)

What Are the Capacities Needed To Put a Particular Intervention in Place?

Intervention-specific capacities are the human, technical, and fiscal conditions important for successfully implementing a particular intervention with quality (Flaspohler et al., 2008; Scaccia et al., 2014)4 . Although table 3 presents global constructs associated with intervention-specific capacity, each new program, practice, or policy has its own set of knowledge and skills required to implement it with quality.

Table 3. Subcomponents of Intervention-Specific Capacity

Subcomponents of Intervention-Specific CapacityDefinition of Subcomponent
Intervention-Specific Knowledge, Skills, and AbilitiesKnowledge, skills, and abilities needed for an intervention, such as an understanding of the EBI’s theory of change or skills being taught in curricula.
Program ChampionKey stakeholder(s) who support an intervention through connections, knowledge, expertise, and social influence.
Specific Implementation Climate SupportsExtent to which the intervention is supported; presence of strong, convincing, informed, and demonstrable management support.
Interorganizational Relationships

Relationships between (a) providers and the training and technical assistance (TTA) support system and (b) between different provider organizations that are used to facilitate implementation.

Source: Scaccia et al. (2014)


4 In Scaccia et al. (2014), this is referred to as “innovation-specific capacity.” We use the term “intervention-specific capacity” synonymously throughout this brief.

Understanding Organizational Readiness as R=MC2

Organizational readiness for an intervention is a function of three components: (1) motivation, (2) general organizational capacity, and (3) intervention-specific capacities. Each component contributes to an organization’s readiness and is abbreviated by R=MC2 (Scaccia et al., 2014). One implication is that if any of the components is zero or near zero, the relationship is multiplicative and the organization is not ready to implement an EBI and any attempt to implement the EBI before the missing element is addressed will likely be unsuccessful.

For organizations to implement EBIs at scale successfully, it is important for funders, researchers, and practitioners to understand readiness and how to create it.

What Does Readiness Look Like During Different Phases of Implementation?

The National Implementation Research Network framework describes four phases of implementation:

  1. Exploration (when people explore the possibility of making use of an innovation)
  2. Installation (where the goal is to acquire or repurpose the resources needed to do the work ahead)
  3. Initial implementation (when the innovation is being used for the first time)
  4. Full implementation (reached when 50% or more of the intended practitioners, staff, or team members are using an effective innovation with fidelity and good outcomes

(National Implementation Research Network, n.d.)

Readiness at one phase does not ensure readiness for the next phase. Communities and organizations need to be ready to enter each phase of implementation: exploration, installation, initial implementation, full implementation. It would be useful for key stakeholders (e.g., leadership, implementers, community members, and clients being served) to discuss readiness regularly. Because readiness is dynamic (i.e., it evolves during a project period and can increase, stay the same, or decrease over time in different phases), these discussions enable project staff to consider readiness components as new challenges arise and to modify the existing approach to meet these needs. Readiness indicators can be assessed through a range of methods, including surveys of providers, clients served, or community members; focus groups eliciting group perspectives; and observation of program components. In the next section, we discuss the implications of the readiness heuristic for staff working in federal agencies.

Implications of Readiness for Staff Working in Federal Agencies

The importance of readiness when implementing an EBI raises a variety of questions for practitioners, funders, training and technical assistance (TTA) providers, and researchers, including:

  • How can funding opportunity announcements (FOAs) measure the readiness of grantees to implement EBIs with quality?
  • How can TTA resources be used to assess and support implementation readiness?
  • How can funders partner with TTA providers and practitioners to enable more organizations to become ready to implement EBIs with quality? Attention to the readiness of grantees during the grant making process and through TTA may improve the likelihood that grantees will succeed in implementing their proposed interventions with fidelity and can achieve the outcomes projected in their applications.

A framework can be helpful in providing a big picture for integrating research and practice to achieve outcomes and to illustrate the roles of practitioners, researchers/evaluators, TTA providers, and funders in this process. Wandersman and colleagues (2008) at the Centers for Disease Control and Prevention and elsewhere developed the Interactive Systems Framework for Dissemination and Implementation (ISF) to help bridge research and practice in innovative ways. The ISF describes three interacting systems: the Delivery System is the organization(s) or community setting that actually implements interventions (e.g., mental health centers, schools). The Delivery System needs sufficient general capacity, motivation, and intervention-specific capacity (i.e., readiness) to be ready to implement an intervention with quality to achieve outcomes. The Support System (e.g., training and technical assistance centers) uses strategies like TTA to strengthen the Delivery System’s ability to be ready to implement interventions with quality (Wandersman et al., 2012). The Synthesis and Translation System synthesizes the products of research and translates them into user-friendly formats that practitioners in the Support and Delivery Systems can easily access and understand. One of the benefits of using a framework such as the ISF to understand readiness is its ability to show how different systems in an organization and community interact to promote readiness. For example, the Support System can provide targeted TTA based on the Delivery System organizations’ readiness needs (e.g., TTA can be provided specifically to improve the motivation of frontline staff). See the sidebar for more information about the ISF and how it is being used to synthesize and translate research to be ready for practitioners’ use and how funders use it to diagnose major gaps in the logic that would lead to outcomes.

Interactive Systems Framework for Dissemination and Implementation (ISF)

  The ISF has been applied in many fields of prevention and treatment, including to summarize the literature; for example, home visiting programs (Paulsell, Del Grosso & Supplee (in press); child abuse prevention (Brodowski et al., 2013), and diagnosing needs for TTA (e.g., a CDC teen pregnancy prevention initiative, Lesesne et al., 2008). Below, we provide a quick example of the three systems and how they interact, drawing upon Lesesne et al. and other developments in teen pregnancy prevention. There are hundreds of articles and multiple evidence-based programs about teen pregnancy prevention. Organizations in the delivery system (e.g., schools, Boys and Girls Clubs) are not likely to ask staff to review the literature. A synthesis and translation of the literature would be helpful. The U.S. Department of Health and Human Services has gathered information on 31 evidence-based programs for teen pregnancy prevention (see http://www.hhs.gov/ash/oah/oah-initiatives/teen_pregnancy/db/programs.html ). However, having access to a website does not mean that the organizations in the delivery system are ready to implement any of the 31 evidence-based programs.

Questions remain, such as “Do organizations have the general capacity to be good host organizations for an evidence-based program? Do they have the innovation-specific capacity required to deliver a specific evidence-based program that meets the requirements for quality implementation, and are they motivated to implement the evidence-based program with quality?” The support system serves as an intermediary system that can provide TTA to help delivery system organizations become ready for implementation.

How To Assess Implementation Readiness Through Funding Opportunity Announcements

In various venues, federal staff have expressed a strong interest in asking about readiness appropriately in FOAs. Table 4 presents sample questions that could be included in FOAs that assess the three readiness components along with criteria that federal staff might use when reviewing FOA responses.

Federal staff could use these criteria in several ways to score applicants. Many of these questions and criteria already exist in the problem statement, project design/goals and objectives, management plans, and management capacity sections of FOAs. Therefore, these questions and criteria could be taken from other sections and included in a new form tied to all federal organizations; they could become a Government Performance and Results Act indicator for organizational functioning. (Readiness would be one part of this.) When scoring, one suggestion is to use a tiered scoring approach in which different applicant scores are associated with different levels of funding, TTA support, and evaluation requirements. Each agency should consider the implications for proposal length and overall scoring when adding this type of information to FOAs.

Table 4a. Examples of Readiness Questions Related to the Motivation Construct To Include in Funding Opportunity Announcements

ComponentProposed Questions Related to MotivationProposed Criteria for Assessment by Federal Agencies and Reviewers Related to Motivation
Relative AdvantageTo what extent does the proposed approach add value to existing practices and programs? Describe how different the proposed approach is from current practices and existing programs and why you think the proposed approach will help you better meet the needs of the youth and communities you serve.

Points given to applicants who describe

  • the programs and practices currently in place and needs not currently being met,
  • how the proposed approach will meet these needs and how it will do a better job than current programs.
CompatibilityDescribe how the proposed approach is compatible with other programs already in your school, organization, or community and the priorities of your school, organization, or community. (E.g., explain how new and current programs and practices will be aligned, including how the new proposed approach will replace or strengthen existing practices, and describe current priorities as outlined in a Strategic Plan or similar document.)

Points given to applicants who describe

  • the current programs and practices (if not yet done in preceding step),
  • how this program will replace another program or how it will be aligned with other programs (e.g., describe how training and Professional Development (PD) for all existing and proposed programs will be coordinated),
  • how the leadership will message, describe, and support this new approach so that it is not regarded as only another program added to everything else that staff are already doing.5
ObservabilityDescribe the extent to which outcomes of the proposed approach are visible (e.g., explain how you will observe the key components of the proposed approach to see whether they are being implemented). These questions are related to a logic model.

Points given to applicants who describe:

  • the different parts of the proposed approach and how it will be observed.6

Table 4b. Examples of Readiness Questions Related to the General Capacity Construct To Include in Funding Opportunity Announcements

ComponentProposed Questions Related to General CapacityProposed Criteria for Assessment by Federal Agencies and Reviewers Related to General Capacity
Organizational InnovativenessDescribe the extent to which your organization is adaptive to change (e.g., provide examples of other recently implemented changes to programs and practices similar to the topic of the grant application; describe staff response to these changes; describe successes or challenges encountered during these changes and how a continuous improvement process helps to deal with challenges that arise).

Points for applicants who describe:

  • the way their organization handled the implementation of a recent change to programs or policies (e.g., how the organization handled an implementation challenge),
  • staff survey results about the organization’s openness to intervention or continuous improvement approaches.
Resource AllocationDescribe the success of your organization at finding and obtaining additional resources (e.g., list grants awarded over the past 5 years; describe grants that you plan to apply for; describe organizational capacity for writing grants).

Points for applicants who:

  • list grants previously awarded and diverse sources of funding (ideally funding would come from different private and public funders, foundations, or community organizations),
  • have a dedicated grant writer or contract writer,
  • describe how they find out about new grants.
LeadershipDescribe the degree to which leaders in the organization support organizational activities and how they plan to convey this support to other staff in the organization.

Points for applicants who describe

  • how leadership supports organizational activities and plans for communicating this support to other staff. For example, does leadership send out monthly updates about organizational activities? Do data collection, monitoring, and annual reviews of organizational activities occur?
StructureHow do organizational structures support the functioning of the organization on a day-to-day basis? (E.g., describe staff size, background and experience, training provided to new staff, opportunities for staff collaboration, time allocated for staff planning and problem solving, the amount of collaboration among staff, and internal decision making processes.)

Points for applicants who describe

  • the way their organization functions on a day-to-day basis (i.e., what training is provided for new staff, how much, when, what structures will be put in place for people to collaboratively plan or solve common issues)
  • staff survey results about collaboration among staff or the effectiveness of decision making processes.

Table 4c. Examples of Readiness Questions Related to the Intervention-Specific Capacity Construct To Include in Funding Opportunity Announcements

ComponentProposed Questions Related to InterventionSpecific CapacityProposed Criteria for Assessment by Federal Agencies and Reviewers Related to Intervention-Specific Capacity
Having a Program ChampionDo key people in your organization support the proposed approach? Who are these people (e.g., role, level)?

Points for applicants who describe:

  • program staff (especially managers and leaders) who have requested this program and who will support it going forward (e.g., what part of the organization these people are in, and how they will continue to be champions of this work throughout the planning and implementation phases).
Implementation ClimateWhat resources will be devoted to the proposed approach? (E.g., Will an implementation team oversee the implementation of the proposed approach? Who will be included in this? How often will they meet? How will the organization or community leaders be involved in this team?)

Points for applicants who describe:

  • the allocation of resources (not only dollars but the number of part- and full-time staff that will be included in the proposed approach),
  • the implementation team (who will be part of this), and
  • plans to hold regular (e.g., biweekly or monthly) meetings with this team.
Interorganizational RelationshipsDo you plan to establish any relationships with other organizations also interested in the proposed approach? If so, please describe who these organizations are and how you plan to maintain such relationships over time.

Points for applicants who describe:

  • how they will share best practices and learn from other organizations implementing a similar approach over the project period (e.g., a description of a plan to collaborate through monthly, quarterly, or yearly calls and visits, sharing reports and data tools).

5 For example, if an organization already has an established mentoring structure in place for new staff and is proposing to implement a new substance abuse program that all staff will be trained in, an applicant could describe how the proposed approach would build on that mentoring structure. For example, if all new staff are required to attend monthly 1-hour PD sessions, one topic of a PD session could be on the new substance abuse program.

6 Depending on what the proposed approach includes, this could be described as (a) weekly staff meetings for groups of people being trained to allow time for problem-solving implementation challenges as they arise, or (b) monthly data meetings for organization staff to review program data and make appropriate changes to the program.

What Is the Role of TTA in Supporting Implementation Readiness?

TTA providers need to use their limited resources and capacities strategically. Analysis of readiness (specifically motivation, general organizational capacity, and intervention-specific capacity) can help to align the strengths and limitations of a grantee with the types of TTA that are deemed appropriate (Wandersman, Chien, & Katz, 2012). Conversely, organizational staff can assess the readiness of their organization to offer different types of support strategies (e.g., online vs. in-person, capacity to train all staff in an organization at once vs. capacity to train a select group of people in multiple cohorts) and develop their implementation plan accordingly. For example, TTA providers could ask organizational staff to answer a brief set of questions such as, “Will there be an implementation team to oversee the implementation of this proposed approach? Who will be part of this team? What kind of training do members of this team need to be successful at implementing this approach? Are there opportunities for ongoing staff collaboration after the training that can be used to reinforce and apply skills learned?” The TTA being provided could then be customized based on responses to these questions (e.g., if the assessment indicates that general capacity needs to be built, then tools, training, and technical assistance (TA) will be used to enhance that specific need). If a TTA provider is working with a site for an extended period of time, readiness assessments could be completed several times during a project and two-way conversations between the TTA provider and TTA recipients could inform how to improve different aspects of readiness.

How Do We Partner With and Foster Readiness in Settings With Limited Capacity and Willingness?

Some organizations with critical needs and who receive federal funding may not currently have the capacity to implement EBIs with quality.7 This problem can be addressed by working with these organizations to first build their capacity through a readiness development phase of the initiative. For example, the Substance Abuse and Mental Health Services Administration (SAMHSA) has done this with its Circles of Care program, which helps tribal communities build a capacity to implement systems of care and in so doing, to compete for grants under the Comprehensive Community Mental Health Services for Children and Their Families Program. The Maternal Infant and Early Childhood Home Visiting Program is another example. Lessons learned from this effort emphasize the importance of developing trusting relationships in which federal staff and TA providers work closely with grantees during the project period to provide support in three major areas: (a) developing a needs assessment and plan for responding to identified needs, (b) developing and carrying out their implementation plans, and (c) designing and carrying out a locally driven rigorous evaluation.

Another example includes the Permanency Innovations Initiative (PII) which is a multi-site federal demonstration project designed to improve permanency outcomes among children in foster care who face the most serious barriers to permanency (Permanency Innovations Initiative Training and Technical Assistance Project & Permanency Innovations Initiative Evaluation Team, 2013).

An overarching objective of PII is to develop interventions that are supported by solid evidence of effectiveness and are ready for replication, adaptation, and broad-scale rollout. PII grantees are guided through four implementation stages: exploration, installation, initial implementation, and full implementation. A specific focus of PII is on building readiness during the exploration stage where grantees receive technical assistance to coordinate a teaming structure, select and promote buy-in for an intervention, and plan for implementation and evaluation of the intervention. These activities are intentionally intended to create readiness for change within grantee organizational structures, ensure the appropriateness of the selected target population for intervention, assess the needs of the target population, and determine the feasibility of the interventions meeting those needs.


7 An important first step is to assess the potential of organizations to develop readiness (DiMaggio & Powell, 1983; Levin & Minton, 1986). Several measures to be used for this purpose are listed in Additional Resources.

Maternal, Infant, and Early Childhood Home Visiting Program

Visiting Program (MIECHV), administered by the Health Resources and Services Administration (HRSA) in collaboration with the Administration for Children & Families (ACF), implemented a provision of the Patient Protection and Affordable Care Act (2010), which expanded home visiting to states, territories, and tribes (Supplee, et al., 2013). Through this effort, three percent of funds were set aside for tribes, tribal organizations, and urban Indian organizations and administered by ACF. To promote the capacity of the tribal MIECHV grantees, ACF required grantees to develop a needs assessment to assess their capacity to implement home visiting programs and use the results to create a plan for responding to identified needs. Ongoing TA and support continued throughout the implementation process by having ACF staff and TA providers work closely with the grantees as they developed and carried out their implementation plans.

Tribal grantees were also required to conduct a rigorous evaluation of their home visiting program, its implementation, or both, because of limited evidence in the field about the http://homvee.acf.hhs.gov/Tribal_Report_2012.pdf http://homvee.acf.hhs.gov/Tribal_Report_2012. The Office of Planning, Research & Evaluation (OPRE) within ACF funded the Tribal Home Visiting Evaluation Institute (TEI) to provide TA in designing locally driven, rigorous evaluations of home visiting. The TEI provides individualized, culturally relevant TA that empowers grantees to conduct research and evaluation that is meaningful for the tribe and meets FOA requirements for rigor (i.e., credibility, applicability, consistency, and neutrality). The TEI builds on ACF’s past and continuing efforts to build evaluation and research capacity within American Indian/American Native communities to improve their early childhood programming. The approach TEI takes to helping grantees develop an evaluation plan that links their community needs to the intended impact of the home visiting model enables communities to build readiness within a prescribed framework.

Directions for Future Research

Although scholarly work has identified several core components of readiness and this brief presented some policy implications, gaps still exist in the field’s understanding of how readiness relates to scaling up and implementing EBIs. Future research directions could include (a) how to measure the relative importance of motivation, general capacity, and intervention-specific capacity in grant applications (and their subcomponents) and (b) how to understand differences in readiness based on scale of implementation (e.g., communities vs. states) and different types of organizations (e.g., child welfare vs. substance abuse treatment providers, faith-based coalitions vs. not-for-profit organizations, small businesses vs. national providers, or urban vs. rural settings). One future direction identified by our federal coauthors involves the analysis of performance data (e.g., Government Performance and Results Act indicators) collected in cross-site, national, and local evaluations, and how readiness constructs can be investigated using these data. This type of program data, which is widely available, should be assessed for its quality. If the quality of the data is confirmed, they could be used along with quantitative and qualitative data collected during the implementation and evaluation of an EBI to increase our understanding of how different aspects of readiness play out in practice. A second future direction involves developing some readiness criteria among different programs that could be useful to explore differences in readiness and its importance in the successful implementation of EBIs. For example, future work could identify criteria for working with communities that have limited infrastructure—in other words, criteria that communities should meet before being selected for implementing an EBI. Based on information gleaned from several federal staff with expertise in this area, these criteria might include (a) strong leadership support from a senior and midlevel staff member, (b) alignment of the intervention with their strategic plan, and (c) evidence of organizational sustainability (e.g., resources are in place for the organization to function for the next 5 years).

Knowledge Sharing Among Staff at Federal Agencies about Readiness

Several approaches to practice and intentional research could enhance knowledge on these topics. Although there has been a focus on linking science with practice across agencies, grant programs, and grantees, this effort has been sporadic. An assessment of FOAs across federal agencies could help ensure the proper exploration of commonalities and differences in effective strategies and performance measurements across varying types of applications (e.g., intervention programs vs. surveillance systems). Research could compare the effectiveness of TTA being provided based on an organization’s readiness by assessing the relationship of TTA to program outcomes for communities with different levels of initial readiness. This could inform best practices for infusing readiness based on certain organizational or community characteristics.

Conclusion

To increase the number of grantees that will succeed at implementing EBIs and achieving positive outcomes, it is important to understand what grantees should have in place before and during implementation. Scholarly work suggests that in order to be successful grantees should be willing (motivated) and able (have the general and intervention-specific capacities). Recognizing the importance of readiness has several policy implications for federal funders; these include assessing readiness during the grant application process, using readiness data to deploy TTA strategically, and building readiness in organizations with limited capacity. More communication, collaboration, and systematic evaluation among federal partners about these and other policy implications can help deepen the evidence base on readiness and promote the successful implementation of EBIs.

Additional Resources

Measures of Organizational Readiness (2005)
 
This http://www.nri-inc.org/reports_pubs/2005/EBPMeasuresOrgReadiness2005.pdf, developed by the National Association of State Mental Health Program Directors (NASMHPD) Research Institute, reviews measurement instruments that can be used to assess the readiness of organizations to implement EBIs.
 
Dimensions of Organizational Readiness (Hoagwood, 2003)
 
The Dimensions of Organizational Readiness (DOOR) instrument was developed as a state planning tool designed to identify stakeholder beliefs and attitudes about organizational processes. This instrument incorporates six domains, including invention characteristics, practitioner characteristics, client characteristics, service delivery characteristics, service agency characteristics, and service system characteristics.
 
General Organizational Index (GOI) (Lynne, Finnerty, & Boyle, 2005)
 
The GOI instrument measures a set of organizational characteristics related to the capacity of organizations to implement and sustain EBIs. It incorporates the following domains: program philosophy, eligibility/client identification, penetration, assessment, individualized treatment plan, individualized treatment, training, and supervision. The GOI score sheet and protocol can be found on pages 57–72 of the SAMHSA document, http://store.samhsa.gov/shin/content/SMA08-4345/EvaluatingYourProgram-AC...
 
Organizational Climate Measure (OCM) (Patterson, et al., 2005)
 
The OCM instrument was developed to assess aspects of organizational climate that affect the effectiveness of organizations. The instrument incorporates the following constructs: acceptance of new ideas, ability to respond to change, identification of need for change, flexibility in responding to changes needed in procedures, support in developing new ideas, and orientation to improvement and innovation.
 
Organizational Readiness for Implementing Change (ORIC) (Shea et al., 2014)
 
The ORIC measure was developed to assess organizational readiness for change in health care settings. The measure includes two facets, change commitment (i.e., organizational members’ shared resolve to implement and change) and change efficacy (i.e., organizational members’ shared belief in their collective capability to implement a change).

References

Brodowski, M. L., Counts, J. M., Gillam, R. J., Baker, L., Spiva Collins, V., Winkle, E., et al. (2013). Translating evidence-based policy to practice: A multilevel partnership using the Interactive Systems Framework. Families in Society: The Journal of Contemporary Social Services, 94(3), 141–149.

DiMaggio, P. J., & Powell, W. W. (1983). The iron cage revisited: Institutional isomorphism and collective rationality in organizational fields. American Sociological review, 48, 147-160.

Flaspohler, P., Duffy, J. L., Wandersman, A., Stillman, L., & Maras, M. (2008). Unpacking prevention capacity: The intersection of research to practice models and community-centered models. American Journal of Community Psychology, 41, 182–196.

Hoagwood, K., Schoenwald, S. K., & Chapman, J. E. (2003). Dimensions of Organizational Readiness-Revised (DOOR-R). Unpublished instrument.

Kratochwill, T. R., & Shernoff, E. S. (2003). Evidence-based practice: Promoting evidence-based interventions in school psychology. School Psychology Quarterly, 18(4), 389.

Lesesne, C. A., Lewis, K. M., White, C. P., Green, D. C., Duffy, J. L., & Wandersman, A. (2008). Promoting science-based approaches to teen pregnancy prevention: Proactively engaging the three systems of the interactive systems framework. American Journal of Community Psychology, 41(3-4), 379-392.

Lewin, A. Y., & Minton, J. W. (1986). Determining organizational effectiveness: another look, and an agenda for research. Management science, 32(5), 514-538.

Lynne, D., Finnerty, M., & Boyle, P. (2005). Implementing EBPs: Early results and the use of the General Organizational Index and other tools in implementing EBPs. Paper presented at the Fifteenth Annual NRI Conference on State Mental Health Agency Services Research, Program Evaluation, and Policy. Baltimore, MD.

 
 
Nevid, J. S. (2011). Psychology: Concepts and Applications: Concepts and Applications. Cengage Learning.

Patterson, M., et al. (2005). Validating the organizational climate measure: links to managerial practices, productivity and innovation. Journal of Organizational Behavior, 26, 379-408. Retrieved from https://wiki.uio.no/admin/amunder/images/a/a0/OCM-1-.pdf

Paulsell, D., Del Grosso, P., & Supplee, L. (in press). Supporting replication and scale-up of evidence-based home visiting programs: Assessing the implementation knowledge base using the Interactive Systems Framework for Dissemination and Implementation. American Journal of Public Health.

Permanency Innovations Initiative Training and Technical Assistance Project & Permanency Innovations Initiative Evaluation Team. (2013). The PII approach: Building implementation and evaluation capacity in child welfare (Rev. ed). Washington, DC: U.S. Department of Health and Human Services, Administration for Children and Families, Children’s Bureau, and Office of Planning, Research and Evaluation.

Scaccia, J., Cook, B., Lamont, A., Wandersman, A., Castellow, J., Katz, J., & Beidas, R. (in press). A practical implementation science heuristic for organizational readiness: R = MC2. Journal of Community Psychology.

Shea, C. M., Jacobs, S. R., Esserman, D. A., Bruce, K., & Weiner, B. J. (2014). Organizational readiness for implementing change: a psychometric assessment of a new measure. Implementation Science, 9(1), 7-22.

Supplee, L. H., Harwood, R. L., Margie, N. G., Meyer, A. L. (2013). New opportunities and directions in home visiting research and evaluation. Zero to Three, 33(3), 45-50.

Wandersman, A., Chien, V. H., & Katz, J. (2012). Toward an evidence-based system for innovation support (tools,) for implementing innovations with quality: Tools, training, technical assistance, quality assurance/quality improvement. American Journal of Community Psychology, 50(3-4), 445–459.

Wandersman, A., Duffy, J., Flaspohler, P., Noonan, R., Lubell, K., Stillman, S., et al. (2008). Bridging the gap between prevention research and practice: The Interactive Systems Framework for Dissemination and Implementation. American Journal of Community Psychology, 41(3–4), 171–181.

Product Type
ASPE Issue Brief