The evaluation of program implementation and effectiveness is an important component of the MFS-IP initiative. RTI International was contracted by the Office of the Assistant Secretary for Planning and Evaluation (ASPE) to conduct a seven-year national evaluation. The evaluation began in October 2006, with the first year focused on evaluation design and documentation of program start-up. The overall goals of the national evaluation of MFS-IP are to describe program implementation in all 13 sites and to determine the impact of programming in a smaller subset of sites. Specifically, the impact evaluation component of the MFS-IP national evaluation will assess whether delivering marriage education programming within the context of particular correctional settings are successful in fostering healthy marriages, strengthening families, and easing community re-entry.
-
Evaluation Framework
-
Although each of the sites is unique, RTI has developed a framework for the evaluation that can accommodate site variation (Figure 1). The framework first isolates inputs as the key factors necessary to implement MFS-IP programming at each site; these include such things as ACF funding, technical assistance, and federal grant requirements. Inputs also include the partnerships grantees are able to form and existing resources external to the site that, in combination with site specific resources, yield a comprehensive set of services and programs available to the program participants (throughputs). The resulting outputs are the number of clients (both the incarcerated individuals and their partners) served by the program, the dosage of services received by program participants, systems-level change affected by the program, and sustainability strategies implemented. The measurement of the changes in knowledge and behaviors (outcomes) for participants and their children is an important factor in the evaluation framework. A variety of key intermediate and long-term outcomes are measured in the evaluation. Finally, as illustrated in the framework, community and individual participant characteristics influence the throughputs, outputs, and outcomes.
Figure 1.
MFS-IP Evaluation Framework
-
-
Evaluation Components
-
The evaluation has been designed to answer the following research questions:
- What are the defining characteristics of the grantees and their programs?
- What lessons can be learned from program implementation?
- How successful were the programs at achieving the desired outcomes?
- To what extent do the interventions appear to have a positive impact?
In addressing these questions, the evaluation includes an implementation assessment and an impact evaluation.
The implementation study primarily assesses the inputs, throughputs and outputs of the projects. It will characterize all 13 programs along a variety of dimensions, including program design, contextual factors shaping program development, project structure, target population, service model, interagency collaborations, and challenges and lessons learned. The primary sources of data for the implementation assessment are in-person interviews with local stakeholders (conducted through annual site visits) and administrative data.
The impact study, which will be conducted in five of the 13 sites, will assess the effectiveness of the MFS-IP programming on participant outcomes. These include marital stability, relationship quality, positive family interactions, family financial well-being, and recidivism. Outcomes will be assessed through longitudinal data collection for both the incarcerated individuals, their partners and through administrative data matching. The impact study will include approximately 2,000 couples across the five sites. Initial in-person survey data collection will take place at baseline. Additional interviews are planned for nine months and 18 months after baseline. There is a potential for a fourth wave of data collection at approximately 25-27 months. The interview will gather detailed data on the background characteristics (e.g., demographics, attitudes, motivation, criminal history, relationship history) and service provision (e.g., types of services received, delivery format, number and duration of sessions). In addition, at each interview point, outcomes will be assessed along several dimensions, including
- relationship quality and stability,
- parenting and child well-being,
- employment and economic stability, and
- criminal behavior and substance abuse.
Impact sites will be chosen using the following criteria: - Sufficient program intensity and provision of couple-based relationship services
- Sufficient projected enrollment to support a treatment and control group
- Programs stage of implementation indicates a stable program design
- Evaluation design possibilities, such as opportunity for random assignment or the identification of an appropriate comparison group, are strong
A site-specific study design will be developed for each impact site.
The analyses of the impact data will focus on differences in intermediate and long-term outcomes between treatment and comparison or control couples. The role of individual- and couple-level factors on outcomes will be considered, as well as programmatic dimensions and other site-level factors that may influence program effectiveness.
-
View full report
"rb.pdf" (pdf, 331.53Kb)
Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®