Oklahoma Marriage Initiative (OMI): Using Research to Guide the Development of an Evolving Statewide Initiative

As a pioneer in broad-based marriage initiatives, the Oklahoma Marriage Initiative (OMI) has charted new territory. Recognizing that there was little prior information to guide implementation designs and strategies for pursuing its goals, OMI planners enlisted the help of a range of research experts from around the country. These experts did not necessarily have ready answers to the challenges faced by the OMI, but were willing to help analyze emerging issues and provide input based on the best available information. This advisory panel has remained engaged since the OMIs beginning, with its function adapting to evolving needs. With the panels guidance, Oklahoma was the first state to conduct a statewide survey of its citizens attitudes and behavior with respect to divorce and marriage. Survey findings were used to inform OMI program decisions and to educate Oklahomans about marriage and divorce in their state; they also may be used as a baseline against which to compare later outcomes. As the initiative has unfolded, the OMI has used research to assess and inform progress, continue expansion, and explore outcomes.

 

As the Oklahoma Marriage Initiative was being developed, its planners made a commitment to rely on research to guide its development. Research has been integral to its evolution, starting with the findings on family structure that first stimulated the idea for the initiative, to the development of subsequent strategies and approaches for implementation. Since 2001, the OMI has been guided by a panel of state and national experts on marriage, divorce, and low-income families. This interdisciplinary Research Advisory Group (RAG) includes academic scholars, university-based practitioners and researchers, and policy experts and evaluators who meet annually and sometimes contribute to other OMI research activities throughout the year. Their ongoing activities, including, for example, conducting small-scale studies of OMI programs and assisting in the development of dissemination materials that translate research findings for a broad audience, have two main benefits: they provide data on which the OMI can base continued development and improvement of program operations, and they lend credibility to and build awareness of the OMI within and outside of Oklahoma.

As the initiative has developed and matured, the research advisory group has considered strategies for assessing the effects of the OMIs overall approach on Oklahoma families. Given the interest in obtaining evidence about statewide change, OMI planners have considered an assessment of broad shifts in attitudes toward marriage and reductions in nonmarital childbearing and divorce rates. More recently, OMI staff have begun to sponsor studies of short-term outcomes and participate in evaluations of the long-term impact of OMI services on particular targeted populations. These latter, more rigorous, evaluations are assessing the impact of services on the quality and stability of couples marriages and relationships and the well-being of their children. OMI officials also view research as a resource for providing information about families, marriage and divorce to the broader population, as exemplified in the series of tip sheets they have issued.

This brief describes how the OMI has used research in planning, ongoing operations, expansions, and evaluation of outcomes. It also discusses the creation and use of a research advisory group and the development of strategies for incorporating research in the context of a dynamic, evolving, broad-based initiative.

 

"

The OMIs Research Advisory Group: A Different Kind of Expert Panel

The idea of a panel of experts guiding or providing advice for large-scale policy change efforts is not a new one. The OMI expert panel, however, took a somewhat different approach. Because the OMI is an ongoing initiative with ambitious goals, OMI planners felt the greatest need was for guidance that would help it grow, particularly during the first several years. Therefore, the RAG focused primarily on issues related to development, with evaluation of outcomes as a secondary and more long-term goal. Another distinction that has set the RAG apart from other similar panels is the ongoing and active involvement of a highly placed state-level decision maker. In light of the initiatives goal of broad social change, the OMIs management and RAG members have viewed this high-level involvement as especially useful.

Particularly during the early years of the initiative, the RAG focused on contributing to development rather than evaluation.  The OMI was a unique concept, so the RAG focused on how prior related research could inform development of the initiative. OMI leaders believe that this emphasis on development, rather than evaluation of outcomes, contributed to the OMIs forward momentum during its first five years. OMI leadership has used the research advisory group to help develop implementation strategies and to help it think through and address early challenges as they emerged. The RAG also has been engaged in assessing the potential for expansion into new areas, such as programs targeted to TANF or Medicaid recipients.

The active involvement of a key policymaker has promoted research-informed decisions about new directions for the initiative.  Since its inception and continuing today, the annual RAG meetings have been attended by the states Secretary and Director of the Department of Human Services (DHS)-the agency that sponsors the OMI. Secretary Howard Hendrick brings many years of experience and special knowledge of Oklahomas families as well as the political and policy context. As the initiatives key decision maker, he has actively proposed and responded to new ideas, engaging in lively discussions with RAG members. This exchange of ideas and information among decision makers, implementers, and researchers has had the effect of bringing research to bear on new directions of interest to the state, and has promoted the initiatives development.

The RAG has served as a sounding board for new ideas, supplied new information, and provided credibility to the initiatives efforts.  During its annual meetings, RAG members have provided input on research that may be relevant to OMI goals and implementation design. The members also have responded to presentations by Public Strategies, Inc. (PSI), the firm that manages the OMIs operations, on the current state of progress and new directions that may be proposed by OMI leadership. RAG members have proposed new ideas and areas in which the initiative might expand or improve. Beyond its substantive contributions, OMI leaders believe the RAGs ongoing involvement has given credibility to the initiatives efforts. In their view, the active participation of nationally known experts and scholars has demonstrated the initiatives commitment to using the latest research to continuously refine the initiative.

A Baseline Assessment of Oklahomans Marital Attitudes and Behavior

One of the first activities of the RAG was to develop a survey of Oklahomas citizens, focusing on attitudes and behavior regarding marriage and divorce. In 2001, the OMI contracted with the Oklahoma State University Bureau of Social Research (OSUBSR) for the statewide survey, which consisted of a random sample of 2,020 adults in the general population and 303 Medicaid clients. The survey was designed in collaboration with members of the RAG, who also contributed to the data analysis and report (Johnson, et al. 2002). Although the motivation for the survey was to assess Oklahomans openness to a statewide marriage initiative and attitudes about marriage and divorce, it ultimately served other purposes as well. In particular, it helped inform OMI planners about key population groups that might be targeted. For example, one finding from the survey was that the average age of first marriage was lower in Oklahoma than the rest of the country, a fact that suggested the implementation of services for young population groups, such as high school students. An oversampling of low-income individuals revealed that although many were single, they (like other Oklahomans) would be interested in marriage education services. This finding contributed to a focus on services for the low-income population.

Using the OMI baseline survey data, several RAG members collaborated on an analysis of the association between participation in premarital preparation and marital quality and stability (Stanley et al. 2006). Because this analysis was based on data collected prior to widespread implementation of OMI workshops and is nonexperimental in nature, it does not assess the impact of the OMI, but confirms the potential usefulness of premarital preparation for Oklahomans. Since Oklahoma conducted its statewide survey, other states, including Florida, Louisiana, and Utah have followed suit.

Monitoring Implementation Progress

To track progress toward statewide saturation, the OMI team developed a management information system to collect and maintain data on workshops and workshop participants. The OMI uses this information to monitor progress throughout its service delivery system, and to update the RAG on the foregoing years activities. Although not designed for research purposes, the data also shed light on such questions as which among the education, faith, social services, corrections, and health sectors are most active, what proportion of the states population has participated in a workshop, the geographic distribution of workshop activity, and how many workshops volunteers tend to lead.

• Tracking Workshop Productivity.  The OMI uses a web-based management information system to record and maintain information about trained workshop leaders and the workshops they conduct. Workshop leaders are asked to enter (or mail in to PSI) information about workshops they plan to conduct or have completed, such as the date, location, and number of participants who complete the workshop. Because most workshop leaders are volunteers, reporting on services can present special challenges. The OMI contracts with OSUBSR to conduct an annual telephone survey of workshop leaders to verify this information and inquire about areas in which leaders may need additional resources or assistance.
• Collecting Information about OMI Workshop Participants.  Obtaining reliable information on OMI workshop participants has been more difficult. In 2003, the OMI developed a short form, to be completed by participants, requesting data on gender, age, education, income, race/ethnicity, and relationship status; information on how they heard about the workshop; and if relevant, the nature of any prior experience with marriage and relationship education. Workshop leaders are not required to ask participants to complete the forms, but are encouraged to do so. Reporting has been inconsistent within and across sectors, resulting in information that is unlikely to be representative of the population of OMI workshop participants. Some workshop leaders suggest that nonresponse may be connected to participants concerns about confidentiality. However, despite asking that participants report only a nickname rather than full identifying information, nonresponse remains high.

Informing the Implementation and Development of Services for Specific Populations

As the initiatives operations have unfolded, other data collection activities have been carried out to inform the development and expansion of OMI services. Intended primarily for internal planning and decision making, projects were undertaken to learn what motivates individuals to participate in services and to better understand the needs and circumstances of special population sub-groups, such as Medicaid recipients, TANF recipients, and prison inmates.

• 2003 Help-Seeking Project.  In 2003, the OMI sponsored the Help-Seeking Project to assess the dynamics around why couples choose to attend marriage education or other couple services (Fournier and Roberts 2003). The project explored barriers that limit a couples willingness to attend targeted services with the intention of informing OMI efforts to bolster attendance at workshops. The sample consisted of a cross-section of Oklahomans with an oversample of Medicaid recipients, a population with a potentially greater need for OMI services. Data collection involved a brief telephone survey with respondents and an expanded format for a sub-set of the sample. The results suggested strategies that might help couples decide to participate in OMI services. Initial findings from the Help-Seeking project served as the basis for a broader study funded by the Administration for Children and Families in the U.S. Department of Health and Human Services that looks at recruitment challenges among low income couples for marriage education programs.
• 2003 Pilot Study of PREP® in Correctional Centers.  In 2002, the Oklahoma Department of Corrections (DOC) collaborated with the OMI to develop a marriage and relationship education program for inmates in state prisons. After training chaplains in the PREP® curriculum, a pilot program was implemented in three correctional facilities in 2003. Two womens facilities and one mens prison were selected across a range of security levels; 80 individuals participated in the pilot. The pilot experience was documented through pre- and post-participation assessments for each participant, and through a small-scale process study. Review of the pilot findings led the Department of Corrections to consider several issues related to expansion of the program to other prisons, including which populations and facilities to include, how to serve married and engaged inmates, and how to address frequent prison transfers that could disrupt weekly participation. Ultimately, the DOC named PREP® as an official program, meaning that prisons with nonvolunteer chaplains must offer the workshop at least once annually.
• 2005 Survey of Medicaid Recipients.  The OSUBSR conducted a survey of pregnant Medicaid recipients in the state to inform the potential development of OMI services for low-income expectant couples. This survey included questions taken from the Fragile Families and Child Well-Being Survey, in which Oklahoma had not been included in the sample. The goals of the survey were to describe the characteristics of pregnant Medicaid recipients; develop an understanding of their relationship quality; explore their choices concerning marriage; assess their interest and motivation in attending marriage education; and identify needed intervention services. The OSUBSR first surveyed Medicaid mothers receiving services associated with pregnancy, and then contacted their babys father. Approximately 500 women and 300 men participated in the survey. Findings helped guide the development of Family Expectations, an OMI program intended to strengthen the relationships and marriages of low-income couples expecting a baby.
• 2005 Survey with TANF Recipients.  In 2005, the OMI conducted a survey of a small number of TANF recipients to inform the development of relationship-focused services for low-income individuals. The survey asked respondents about their attitudes and beliefs about romantic relationships, their current relationship status and the quality of and aspirations for their current relationship. While this effort did not result in a formal report or set of findings, survey responses assisted OMI planners and curriculum developers in understanding this target population, and informed the development of an official PREP® adaptation for low-income single parents, known as Within My Reach.

Nonexperimental Evaluations of Short-Term Outcomes in Specific Programs

Because of the diverse range of populations served and the very specific venues in which programs operate, the OMI can be considered a laboratory for exploring program strategies that, although drawing on research for their design, have not previously been tested. First steps toward understanding their potential effects on families included OMI-sponsored studies of implementation and outcomes in two programs - in Oklahomas prisons and in retreats for couples with adoptive and foster children.

• Prison Inmates.  In 2004, the curriculum developers and others conducted a study with prison inmates participating in PREP® classes at Oklahoma prisons. Prison chaplains collected self-report questionnaires completed by inmates immediately before the first class and upon completing the last class (Einhorn et al. in press). Data were obtained from 448 inmates, although the researchers restricted analysis of the data to the 254 participants who completed questionnaires and were also in the same relationship at both data collection points. The reported results thus omit participants who might have experienced more problems. The participants included in the findings reported positive changes on a variety of dimensions associated with relationship quality, both among overall participants, and for key racial and ethnic subgroups. This outcome research is considered a first step toward potential future work that would more rigorously evaluate the effect of PREP® for prison inmates and their partners.
• Adoptive/Foster Parents.  As part of its grant from the Administration for Children and Families at the U.S. Department of Health and Human Services, the developers of the ENRICH inventory and others are assessing the short-term outcomes for couples who participate in the OMIs adoptive/foster parent retreats. At the beginning of each retreat, couples complete the ENRICH questionnaire, an in-depth inventory of each persons attitudes toward long-term marital stability and perceptions of marital satisfaction, and their ability to communicate and problem-solve. At the end of the retreat, couples complete a feedback form, reporting on perceptions of change they may have experienced during the weekend retreat. Follow-up interviews are conducted with couples six months later, with open- and closed-ended questions about changes in their relationship since the retreat.

Experimental Evaluations of Programs for Expectant Couples

The OMI is part of a demonstration program in two national multi-site evaluations funded by the Administration for Children and Families at the U.S. Department of Health and Human Services. As such, it is helping to generate estimates of the impact of marriage education on a targeted group of unmarried and married couples involved in controlled experimental studies.

• Unmarried Couples: Building Strong Families.  Through its Family Expectations program, Oklahoma is one of seven states participating in the Building Strong Families demonstration and evaluation, which is being conducted by Mathematica Policy Research (MPR). The demonstration enrolls low-income, unmarried couples who are currently expecting a child or have a newborn up to three months old. Eligible couples are randomly assigned to the program or to a control group, and MPR conducts follow-up interviews with both parents in both groups at 15 months after random assignment and again when their child is three years old. Outcomes of the program and control groups will be compared to assess program impacts on the status and quality of the couple relationship, parental well-being and co-parenting, and family and child outcomes.
• Married Couples: Strengthening Healthy Marriage.  Also through its Family Expectations program, Oklahoma is a site for the Supporting Healthy Marriage national demonstration and evaluation, led by MDRC. This demonstration targets economically disadvantaged married couples and their children. Like Building Strong Families, the evaluation employs an experimental design using random assignment to a program or control group. Plans call for follow-up assessments at 12, 36, and 60 months after random assignment. The study will assess the impact of the intervention on relationship processes, marital stability and quality, attitudes toward marriage, child well-being and development, and economic outcomes.

Assessment of Broad Social Change

There are numerous obstacles to rigorous evaluation of statewide impacts. Chief among these challenges is the lack of a counterfactual, that is, another state that is the same in every way except for presence of the OMI, whose social outcomes could be compared to those observed in Oklahoma. One alternative approach is to compare Oklahomas current divorce and nonmarital childbearing rates to measures taken prior to, or at the beginning of, the initiative. While this pre-post method would provide valuable information, it would not determine the extent to which observed changes occurred as a result of the OMI as opposed to other factors, such as changes in the economy. Certain evaluation techniques, however, may be developed and applied to reduce these concerns. The design of such an evaluation would require special attention and tailoring to the OMIs unique circumstances.

A second important issue to consider in any assessment of state-level outcomes is that broad social change in the culture of marriage is likely to take some time to emerge, regardless of the quality and depth of OMI program implementation. The pervasiveness of divorce and nonmarital childbearing suggests that changes in deep-seated values and attitudes are not likely to occur quickly or easily. It is possible that assessments of state-level outcomes of divorce and nonmarital childbearing may not capture change until the OMIs reach extends well beyond current implementation.

Assessing cultural change in attitudes and norms about marriage will similarly require careful thought about research design. It is possible to design and conduct a survey to assess change in knowledge and attitudes, such as whether more people think healthy marriage is something that can be learned, and OMI planners have begun to take steps in that direction. For the same reasons described above, such an approach would not permit observed changes to be confidently attributed to the influence of the OMI, but it may be possible to design a quasi-experimental approach to reduce threats to the validity of findings.

Disseminating Information About Marriage to the General Public

OMI leadership considers research to be useful not just for informing development, assessing progress, and understanding impacts, but also for educating the public about relationships and marriage. The RAG has engaged in developing practical information about marriage and divorce derived, in part, from its statewide baseline survey, for distribution to Oklahoma families. This effort resulted in a series of information sheets that distills research into consumer-friendly questions and answers, addressing common questions about couple relationships.

Each single-page tip sheet focuses on a topic of interest to couples who are considering marriage or are already married, often pointing out or debunking common misconceptions about marriage and divorce. Examples of topics include: how cohabitation correlates to later success in marriage; marriage and age; how divorce affects children; what a healthy marriage is; the value of premarital education; how to decide whether you and your partner are ready for marriage; identifying an unhealthy relationship; and how having a child changes marriage. These tip sheets are released at regular intervals and posted on the OMI website (www.okmarriage.org). OMI managers expect that the information in these documents could be used by individuals, and in a variety of settings and with different audiences, such as in churches, newspapers or newsletters, and public addresses. Workshop leaders also may use these products as supplemental handouts for participants.

Conclusion

The OMIs research advisory group has remained involved in the initiative since its inception, and is viewed by OMI leaders as an essential component of the initiative. Its usefulness is likely linked to two factors: flexibility and synergy.

Flexibility: The RAG has shown flexibility at multiple levels, including its willingness to respond to the initiatives developmental stage and its ability to work across disciplines toward a common goal. RAG members have strived to support the OMI by providing or collecting information for the initial design and the initiatives development, expansion, and evaluation. The ability to work across disciplines toward a common goal has been a hallmark of the RAG, and has resulted in multiple collaborations that often further the research interests of individual RAG members as well as those of the initiative.

Synergy: The composition of the RAG-including researchers, practitioners, and policymakers-has created a synergy that supports the ongoing evolution of the initiative. The annual RAG meetings present an opportunity for the state to emphasize its policy and program interests, for program implementers to bring forward the issues they are confronting, and for experts to contribute their knowledge from research and past experience. The common interests and cohesiveness of the group contribute to the dynamics and staying power of the initiative.

Among the challenges facing the OMI are how to rigorously evaluate the impact of the initiative on state-level outcomes, and how to collect information from workshop participants so that a better understanding of who is participating can be obtained. Future research efforts will likely be designed to address both of these issues.

Bibliography

Einhorn, L., T. Williams, S. Stanley, N. Wunderline, and H. Markman. PREP Inside and Out: Marriage education as an intervention for prisoner reintegration. Denver, CO: University of Denver, in press.

Fournier, David, and Kelly Roberts. Examination of Help Seeking For Couple Relationships in Oklahoma. Oklahoma City, OK: Oklahoma State University Human Development and Family Science, 2003.

Hershey, Alan A., Barbara Devaney, M. Robin Dion, and Sheena McConnell. Building Strong Families: Guidelines for Developing Programs. Washington, DC: Mathematica Policy Research, Inc., 2004.

Johnson, Christine, Scott Stanley, Norval Glenn, Paul Amato, Steve Nock, Howard Markman, and M. Robin Dion. Marriage in Oklahoma: 2001 Baseline Statewide Survey on Marriage and Divorce. Oklahoma: Oklahoma State University Bureau for Social Research, 2002.

MDRC. Guidelines for Supporting Healthy Marriage Demonstration Programs. New York, NY: MDRC, 2005. [PDF - 14 pages]

Stanley, Scott, Paul R. Amato, Howard Markman, and Christine Johnson. 2006. Premarital Education, Marital Quality, and Marital Stability. Journal of Family Psychology 20: 117-126.

How to Obtain a Printed Copy

To obtain a printed copy of this report, send the title and your mailing information to:

Human Services Policy, Room 404E
Assistant Secretary for Planning and Evaluation
U.S. Department of Health and Human Services
200 Independence Av, SW
Washington, DC 20201

Fax:  (202) 690-6562