Inside the Black Box of Interactions Between Programs and Participants: Re-conceptualizing Subgroups for Fatherhood Program Evaluation. I. INTRODUCTION


Fathers are critical to their children's well-being and development. Unfortunately, some fathers are not able to provide the consistent care and financial support their children need. Low-income fathers, in particular, are less likely to live with and have contact with their children (Nelson 2004) and may have greater difficulty providing for their children emotionally or financially.

In reauthorizing the TANF program, the Deficit Reduction Act of 2005 (DRA) created the Healthy Marriage and Responsible Fatherhood (HMRF) grant program to promote healthy marriages and foster responsible fatherhood among low-income individuals and couples. DRA authorized $150 million in each of fiscal years 2006 through 2010 for these programs, up to $50 million of which could be used for responsible fatherhood programs. Five-year grants were awarded in 2006 to 94 responsible fatherhood (RF) and 122 healthy marriage (HM) grantees. The Claims Resolution Act of 2010 (CRA) re-authorized this grant program, increasing the focus on economic stability by allowing healthy marriage programs to offer job and career advancement services and broadening its reach by allowing grantees to provide marriage and relationship education to any low-income individual (not just unmarried pregnant women and expectant fathers). CRA also strengthened the emphasis on fatherhood, requiring that funding be equally split between healthy marriage and responsible fatherhood programs. Three-year grants were awarded in 2011 to 55 RF and 60 HM grantees.

The U.S. Department of Health and Human Services (DHHS) is interested in learning more about the effectiveness of such programs, including those aimed at promoting responsible fatherhood and economic self-sufficiency among low-income and noncustodial fathers. In particular, there is growing interest in ascertaining "what works for whom"-that is, in examining program impacts among meaningful subgroups of fathers in order to foster better program design and provide a basis for targeting program services. Toward that end, DHHS's Office of the Assistant Secretary for Planning and Evaluation (ASPE) seeks to learn more about evidence-based strategies for defining subgroups of low-income men to inform future evaluations of fatherhood programs, thereby maximizing the information that can be gleaned from these evaluations.

In September 2011, ASPE engaged Mathematica to "look inside the black box" and identify psychosocial predictors of behavior change that may be used to create baseline subgroups for use in future evaluations of fatherhood programs. Project tasks include:

·      A scan of innovative approaches that have been used in other fields to examine subgroups.

·      A review of theories of behavior change and of fathering behavior, and a review of the empirical literature on low-income fathers to examine whether and how these theoretically relevant predictors of behavior and behavior change have been studied and found to be predictive of fatherhood-related outcomes.

·      Synthesizing findings in a written report.

·      Convening a roundtable of federal and nonfederal experts to review findings, identify gaps in our knowledge, and discuss effective strategies for incorporating project findings into evaluations of fatherhood programs and initiatives.

We summarize and synthesize study findings in this report. After this brief introduction, in Chapter II, we discuss the utility of studying subgroups in program evaluation research, present an overview of typical approaches to creating subgroups, and propose alternative hypotheses regarding subgroups of individuals for whom programs may be more or less effective. In Chapter III, we provide an overview of study methods. In Chapter IV, we present findings from our scan of subgrouping approaches, providing details on the key concepts and innovative approaches for defining subgroups in other fields of study. In Chapter V, we present findings regarding the theoretical psychosocial determinants of behavior change and integrate these concepts into a coherent framework relevant to the study of fathers. In Chapter VI, we use this framework to present findings from our review of the literature examining these psychosocial factors in samples of low-income fathers. In Chapter VII, we summarize findings from Chapters V and VI. In Chapter VIII, we discuss how these concepts and approaches to subgrouping might be applied to fatherhood program evaluations. And finally, in Chapter IX, we present our conclusions.

View full report


"rpt_insidebb.pdf" (pdf, 1.35Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®