Inside the Black Box of Interactions Between Programs and Participants: Re-conceptualizing Subgroups for Fatherhood Program Evaluation. APPENDIX A: APPROACH TO SEARCHING AND SELECTING STUDIES FOR REVIEW IN ENVIRONMENTAL SCAN OF SUBGROUPING METHODS


The project team developed a standard study review template designed to capture key data from each study. The template was piloted and refined according to feedback from all team members before its use for the reviews. For each study reviewed in the scan, in addition to any identifying information, we documented the following:

·      Subgrouping method

·      Subgroups

·      Specific constructs and variables used to create subgroups

In addition, in preparing for our upcoming review of the literature on the determinants of behavior change (generally and specifically relating to outcomes targeted by fatherhood programs), we identified psychosocial variables that may be used in creating subgroups potentially relevant to fathers and fatherhood programs. (In a subsequent report, we will summarize the findings from a more extensive literature review of these predictors of behavior change.)

The environmental scan consisted of three types of searches: (1) audience segmentation methods used in public health, (2) service-user typology methods used to study homeless and public housing populations, and (3) subgrouping methods used in evaluations of fatherhood programs.

A. Audience Segmentation Methods in Public Health Literature

We consulted the public health research literature on the adoption of social marketing approaches to targeting public health interventions. Our initial database search identified 63 unduplicated studies, of which we screened out 42 because they:

·      Did not mention audience segmentation in the abstract

·      Appeared to focus only on a definition of audience segmentation or recommended audience segmentation in future research rather than describing a particular audience segmentation method

·      Lacked a focus on public health

·      Were published by an author whose more recent work we already have identified/included

We completed reviews of 14 studies for which full-text articles were available and documented the reviews in an online database for analysis.

B. Service-User Typology Methods in Homelessness/Public Housing and Other Literature

We simultaneously consulted the literature on service-user typologies developed to identify subgroups (for example, public housing residents, homeless individuals, low-income mothers, and welfare recipients) that may differ in the nature or extent of their needs for a variety of services. The purpose of the search was to identify subgrouping methods used specifically in service-user typology research and to identify general categories of constructs/variables used to create subgroups, as pertaining to fatherhood-related outcomes.

We began by compiling a list of the 95 references and associated abstracts referenced in Rog et al. (2007; Appendix B) and Theodos et al. (2010). To identify studies focusing on developing service-user typologies for other populations, such as welfare recipients or low-income women, we conducted a database search and added 4 service-user typology articles to our list. Overall, we screened out studies that:

·      Did not discuss subgrouping methods or typologies.

·      Discussed subgrouping methods but described methods or populations for which we already had several sources of information. (For example, we already had several references discussing cluster analysis for homeless individuals.)

·      Discussed predictors or determinants of homelessness, but with a focus inapplicable to fatherhood outcomes.

We identified eight studies that were relevant for review as part of this search.

C. Subgrouping Methods Used in Evaluations of Fatherhood Programs

We examined impact evaluations identified in the Strengthening Families Evidence Review (SFER)-a systematic review of research on programs serving low-income fathers (Avellar et al. 2011) and programs serving low-income couples (Avellar et al. 2012). Specifically, we selected impact evaluations rated as "high quality," then reviewed the original impact reports to see the extent to which subgroup impacts were examined and, if so, which subgroups were considered, how such subgroups were defined, and any impacts found.

We reviewed a total of 12 high quality impact evaluations-eight evaluating fatherhood programs, and four evaluating programs serving low-income couples.

View full report


"rpt_insidebb.pdf" (pdf, 1.35Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®