Oklahoma Marriage Initiative (OMI): A Process Evaluation. Potential for Future Evaluation

05/23/2008

This evaluation has examined the implementation of the OMI, but not the extent to which its ultimate goals have been achieved: namely, reductions in divorce and nonmarital childbearing, and increases in the number of children growing up with their own parents in a healthy marriage.  It also has not rigorously assessed the extent to which societal attitudes and norms have changed with respect to marriage and divorce, factors that could ultimately be associated with broad cultural shifts and behavior change.

A strong assessment of the OMIs impact on divorce and nonmarital childbearing requires a well-matched comparison group.  There are numerous obstacles to rigorous evaluation of statewide impacts.  Chief among these challenges is that there is no counterfactual  another state that is the same in every way except for presence of the OMI, and whose social outcomes could be compared to outcomes observed in Oklahoma.  One approach is to compare Oklahomas divorce and nonmarital childbearing rates to measures taken prior to, or at the beginning of, the initiative.  While this approach would provide useful information, it would not conclusively determine the extent to which any observed changes have occurred due to the OMI or because of other factors, such as changes in the economy.  By making certain assumptions, however, evaluation techniques can be developed and applied to reduce these concerns.  The design of such an evaluation would require special attention and tailoring to the OMIs unique circumstances.

Assessing cultural change in attitudes and norms will similarly require careful thought about research design.  It is possible to design and conduct a survey to assess change in knowledge and attitudes, such as whether more people think healthy marriage is something that can be learned, and the OMI has begun to take steps in that direction.  For the same reasons as described above, such an approach would not permit observed changes to be confidently attributed to the influence of the OMI.  A carefully designed quasi-experimental approach, however, could potentially be brought to bear to reduce threats to the validity of findings.

If changes in marriage, divorce, and nonmarital childbearing occur, they are likely to emerge over the long term.  An important issue to consider in any assessment of state-level outcomes is that broad societal change in the culture of marriage is likely to take time to emerge, regardless of the quality or depth of OMI program implementation.  The pervasiveness of divorce and nonmarital childbearing suggests that changes in deep-seated values and attitudes are not likely to come quickly and easily.  In addition, because nearly 70 percent of participants are youth or single parents, it will take time to observe any effects on divorce.  These individuals are not married, and national statistics indicate that people are marrying later in life.  Once people do marry, the average duration of first marriage today is about seven to eight years (Kreider and Fields 2001).  In light of the unwed status of most OMI participants, it may be possible to detect impacts on marriage and nonmarital childbearing sooner than impacts on divorce.  Still, it is unlikely that assessments of state-level outcomes on divorce and nonmarital childbearing will be able to capture any change until the OMIs reach extends well beyond its current implementation.  

Some elements may be rigorously evaluated through random-assignment experiments.  Another, more rigorous approach to evaluating the OMI involves experiments in which people are randomly assigned to either the program or a control group that does not have access to the program.  The OMI is already engaged in such experiments with particular OMI components.  The OMIs program for couples having a new baby together, Family Expectations, is being studied as part of the national evaluations of Building Strong Families and Supporting Healthy Marriages.  These controlled evaluations, conducted by nationally recognized policy research firms, are assessing the impact of services on the quality and stability of couples marriages and relationships and the well-being of their children.

It would be possible to extend this evaluation method to other OMI programs.  Some programs lend themselves well to experimental designs that test program effectiveness  particularly those that focus on a specific population.  These include, for example, the program for adoptive and foster parents, the Within My Reach program for TANF recipients, and PREP® as it is deployed in correctional centers.  Particularly in light of the high take-up rate of high school classes that contain OMI curricula, Connections-PREP® could be evaluated using a quasi-experimental design.  None of these adaptations has yet been rigorously evaluated for their effects on relationships and marriage.  Obtaining solid evidence of the impact of these programs would either provide the OMI with support to continue and expand its efforts, or would suggest that improvements are needed to achieve the desired outcomes.

Improving management information would foster better assessment of saturation and lay the foundation for future evaluations.  Issues involved in gathering information about workshops and workshop participants could be examined and addressed in order to improve assessment of the extent of statewide saturation and to reveal gaps in coverage.  The lack of basic information about who is being served is a central issue.  For example, although many high school students are being served, it is not known how many of these youths are boys or girls.  While data collection in a system that relies heavily on volunteers has many challenges, the number of couples being served, their marital status, and income levels are important to gauge whom the OMI is reaching, and can inform directions for both future implementation and evaluation.

View full report

Preview
Download

"report.pdf" (pdf, 726.12Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®