Increasing Organ Donation and Transplantation: The Challenge of Evaluation. A. Table of Health Behavior Activity Evaluations


Study Overview Evaluation Performance Indicators Target Findings Comments
Smoking Cessation
Lando 1991 General media campaign tied to a contest to ensure follow-up; Evaluated using telephone surveys to compare those who sent in interest cards and pledged to stop-smoking versus those who sent in cards but did not pledge to stop smoking Nonrandomized trial with contemporane-ous controls, also compared outcomes from previous study (historical control) Outcomes – comparing difference in quit rates between pledgers and non-pledgers Community- wide intervention, public awareness activity Study contained two comparisons: (1) historical control and (2) differences between pledgers. The study found that an extended enrollment period and intensive campaign increased enrollment, and overall quit rates. Pledgers had higher self-reported abstinence rates. No control group of a community without the intervention, but the historical control is a good indicator of change; detects differences in those who quit versus those who did not
Popham 1993 A group of people who quit smoking were selected from the entire population of those exposed to California’s anti-tobacco media campaign to measure the exposure of those who quit to the campaign Cross-sectional study, retrospective, no comparison group Whether or not respondents saw the advertisements;

Attempt to determine if process correlates to the specified outcome

Community, public awareness activity Study sought to evaluate what influenced people to stop smoking during the campaign and determined that the media campaign had influenced change No comparison group for this study; other smokers who didn’t quit smoking may have seen the study and not quit
Werch 1994 Applies Prochaska’s stages of change model to the formation of a unhealthy behavior to better focus prevention strategies; Paper is theoretical without an actual intervention Methods/theory paper N/A Community, Public Awareness N/A Applies health belief, social learning, and behavioral self-control theories to stages of change and strategies that could be used in prevention or behavior promotion; May provide a good theoretical background
Prochaska 1983 Used a test of 40 questions to track subjects for 2 years to determine the progression through stages of change related to quitting smoking Methods/theory paper N/A Community, Public Awareness N/A No evaluation of specific intervention but may be correlated to success in stages of change
Prochaska 1992 Outlines process by which addictive behaviors are modified and stages of change: precontemplation, contemplation, preparation, action and maintenance Methods/theory paper N/A Community, public awareness N/A Stages of change may be a useful aid to evaluate organ donation strategies; a baseline study could determine how people cycle through the stages of change leading to consent; then, interventions could be designed and evaluated based on these stages of change
Mammography and Other Cancer Screening
Strickland 1997 Compared the impact of three interventions (physician message, message with class, message, class and reinforcement) on breast self examination compliance Large randomized controlled trial Percentages of women doing breast self exams at 2 follow-ups over the course of 1 year Provider organization intervention, women general awareness The more comprehensive intervention was more successful in the number of women doing breast self examination One of the few examples of a large RCT done to influence health behavior changes that may serve as a good model for the organ donation world (particularly because of the provider education/support focus)
Champion 1995 An evaluation to measure the effect of an individualized belief and/or information intervention on mammography compliance Randomized controlled trial Stages of change and adherence to mammography screening Control group and 3 intervention groups; educational activity Women in the belief/ information group were 2x more likely to have been compliant with intervention 1 year later; women receiving the belief intervention were more likely to move to a higher stage of change A good example of a randomized, controlled trial used in program evaluation regarding health behavior education with 4 randomized intervention groups, baseline screening and 2 follow-up measurements
Campbell 1997 An office-based computer system was randomized to provide women who filled out a computer health survey with either follow-up recommendations or no recommendations. Study sought to determine if these recommendations made a difference Small randomized controlled trial Compared #s who had a PAP smear in the 6 months following the visit Targeted members of community, public awareness Results were inconclusive The investigators were unable to draw conclusions regarding the effectiveness of the computer system due to the modest proportions of women screened, the small numbers, and the fact that the computer survey may have created an intervention effect in the control group
Worden 1994 Description of a study design methodology based on baseline data to assess women’s participation in mammography, clinical breast examination, and breast self-examination and to measure factors that may affect their participation. Nonrandomized trial with contemporaneous controls Will look at intermediate objectives, behavioral outcomes, and health benefits Women, their physicians, and the health care system in a single, comprehensive program; public awareness activity and provider education and support The telephone survey did not receive as good a response as the household surveys Sets up a useful framework for designing an evaluation of a multi-level intervention; uses baseline survey data to develop program components
Rakowski 1992 Survey to assess whether or not stages of change impacted women’s decision about mammography Survey, one point in time Questions developed to assess stages of change and at what stage certain behaviors were adopted Community, public awareness Provided results that could be used to extend the transtheoretical model of behavior change to mammography screening Provides a model for the kind of baseline study that the organ donation community could use to learn how to measure stages of change. Then actions could be planned from this study.
Rakowski 1997 Survey to examine the utility of the construct of decisional balance for mammography and pap testing Survey Attitudes regarding testing and recent testing behavior Community, public awareness activity Options about testing may carry over for several kinds of testing. The ability to employ combined indicators for frequency of testing and test-related opinions is promising for being able to take a more comprehensive approach to women’s health.
Mayer-Oakes 1996 Longitudinal study of women under the care of internists to determine factors that predict mammography use Telephone survey Sociodemograph-ics, physical functioning, psychosocial functioning, preventive and self-care behaviors and mammography use N/A – no intervention but a survey of women in general Mammography screening was significantly higher among women who had recently received a Pap smear, whose annual household incomes exceeded $30,000 and whose personal health care habits were preventively oriented. A baseline survey from which interventions could be built and evaluated
Savage 1996 Survey to assess the Health Belief Model in relation to a Woman’s decisions to have screening mammographies and do breast self examinations Telephone survey Questions developed to assess correlates of health behaviors including demographics and attitudes N/A – no intervention but a survey of women in general Different variables were found to be predictors of adherence to mammography and breast self exam A baseline survey from which interventions could be built
Scammon 1995 An evaluation of the role of free mammograms in motivating first-time screening in a community. Series of consecutive cases Demographics and attitudes of factors in decision to get mammogram were measured. Women, asymptomatic for breast cancer, 35 and older; public awareness activity The free mammography may have helped bring women in, but other factors need to be reinforced (i.e., the feeling of doing something good) to maintain screening patterns. No controls, and women who were willing to return the survey are the results that were included.
Physical Activity Promotion / Obesity Treatment
Stone 1996 An 8-year multicenter randomized trial investigating whether behaviorally oriented cardiovascular school and family-based health program produced positive changes in the health habits that favorably affect the cardiovascular risk profile of elementary school preadolescents. Large randomized controlled trial (unit of randomization is the school rather than individuals) Outcome measures including changes in serum cholesterol, physical activity, and dietary measures Individual students and family, public awareness activity As of publication, data analysis was in progress. However, due to low prevalence rates of smoking in the 5th grade, the study did not have sufficient power to detect differences between experimental and control groups. This trial uses schools as the unit of randomization – similarly the organ donation community could use hospitals or regional-OPOs as the level of randomization.
Marcus 1997 A pilot study to test the feasibility and efficacy of a physician-delivered physical activity counseling intervention. Small randomized controlled trial Physical activity levels Provider education to improve patient results Physician-delivered physical activity interventions may be an effective way to achieve wide-spread improvements in the physical activity of middle-aged and older adults. This small scale study may be similar to the kinds of hospital-level provider education efforts conducted in the organ donation community.
Cakfas 1997 Evaluated whether an intervention to change physical activity changed such mediators as stages of change, self-efficacy and social support, and whether changes in the mediators were associated with behavior change. Nonrandom contemporaneous control Mediators (changes in stages of behavior) and outcomes (actual changes in behavior) Providers and patient education program Patients who were counseled improved significantly more than those in the control group on behavioral and cognitive processes of change. Two of three mediator variables were associated with changes in physical activity. Serves as a good model for how the organ donation community might test stages of change model, however measuring donations as a result of this intervention might be difficult.
Fontaine 1997 Tested a self-efficacy questionnaire to assess its predictive validity in treatment outcomes for obesity Time-series Outcome measures of attendance and weight loss, intermediate measures of self-efficacy Individual patients, awareness Self-efficacy judgments (as measured by the Weight Efficacy Lifestyle Questionnaire) are not predictive of short-term obesity treatment outcomes Another example of a study used to test the predictive value of measures of propensity to change health behaviors
Marcus 1992 Survey to assess whether or not stages of change and self-efficacy impact exercise behavior change Survey, one point in time Questions developed to assess stages of change and at what stage certain behaviors were adopted Community, public awareness activity Provided results that could be used to extend the transtheoretical model of behavior change to stages of exercise behavior change; basic goal is to develop a reliable instrument Provides a model for the kind of baseline study that the organ donation community could use to learn how to measure stages of change; actions could be planned from this study
Potvin 1997 Documented the differing prevalence rates for stages of change for physical activity across rural, suburban and inner city communities using survey methods and controlling for education, gender and disease status Survey Stage-of-change measurements Community, public awareness The findings suggest that above and beyond individual difference variables, structural components such as type of community are related to people’s readiness for physical activity involvement There may be community-level or other types of factors which influence the stage of change someone is in with regard to organ donation – further studies may be required to determine them
Cardinal 1995 A stage of exercise scale was developed to differentiate between subjects classified into each of the transtheoretical model stages of change and tested in females Develops survey instrument for testing Several measures that might impact stages of change Female adults, public awareness activity Results showed that the scale was able to significantly and meaningfully differentiate between subjects classified by stage in terms of exercise energy expenditure, physical activity energy expenditure and other metabolic measures The organ donation community may want to develop and rigorously test a similar type of scale for measuring the stages of change for organ donation
Cardinal 1995 The study developed, evaluated, and compared two sets of written materials which promoted regular physical activity and fitness-promoting behaviors. Review of materials to promote behavior change Measured whether or not the materials were readable Community, public awareness No significant differences were observed between the two information packets; this could have been due to the small sample size. One method for measuring public reaction to information on organ donation, but instruments would have to be further tested to see which results in actual behavior change.
Marcus 1994 Review of studies which measure stages of change in relation to exercise behavior change Review and theory N/A Community, public awareness activity Applications of the transtheoretical model of the initiation, adoption, and maintenance of exercise behavior from clinical community, and public health perspectives are discussed The transtheoretical theory of behavior change may also be applicable to organ donation
Occupational Health
Jeffery 1993 An evaluation of a program that used tangible incentives to promote worksite health (specifically obesity and smoking cessation) RCT – Quasi Process measures of numbers of participating employees and one outcome measure of average weight loss Community, public awareness activity The power of the study was too small to determine differences between the incentive and non-incentive groups One example of an attempt to conduct a randomized, controlled study in a naturalistic setting, and suggestions for improving study power
Kurtz 1997 An evaluation of peer and professional trainers in a union-based occupational health and safety training program Non-randomized controlled trial Self-efficacy, outcome efficacy, and behavior Community, trainer education program Workers trained in health safety by their co-workers reported changing behavior more often than those workers trained by professional trainers Intervention may be comparable to hospital-level interventions and organ donation education programs; this paper provides one type of evaluation model
Zwerling 1997 Reviews evaluation strategies for programs to prevent occupational injury – notes that these kinds of evaluations could be improved and presents a hierarchy for improvement Few RCTs are conducted – researchers should begin with qualitative studies and follow-up with simple quasi-experimental designs, then more complicated quasi-experimental designs, and RCTs when possible Outcomes in reduced injury and intermediate outcomes of measurable events along the causal pathway Worker education Proposes a number of strategies to improve these evaluations while recognizing that RCTs may not be feasible due to difficulties of randomization, study groups contamination, and population turnover Provides an overview of evaluation strategies in another behavioral health intervention field that may inform efforts to improve evaluations of organ donation activities
Other Conditions
Forst 1990 A review of AIDS programs evaluations noting the lack of methodologically rigorous studies and the expense of longitudinal studies over time. Discusses several evaluation options Ouctome measures should include changes in knowledge, linked to attitude changes, linked to behavior changes – few studies achieve this Community and targeted members Some problems with the evaluations included: sites used only a post-test design to measure competency but no measure of change; tests were not standardized or compared across sites Forst suggests areas for improvements in AIDS education evaluation studies; these programs face many similar difficulties as organ donation activity evaluation
Nonis 1996 A survey was conducted of college students blood donors versus non-donors to identify attitudes and plan future studies Framework study, No real intervention to change behaviors Attitudes College Students Develops strategies for blood collection agencies to market to students Blood donation is more closely related to organ donation than some of the other behavioral change activities—may provide a useful methodology for understanding organ donation tendencies
Evaluation Methods
Terrin 1997 Paper evaluates the three important approaches to evaluating lifestyle and health outcomes: observational studies, individual subject random assignment trials, and community random assignment trials Methods paper for community-level interventions Observational Studies, individual subject random assignment, and community random assignment Lifestyle and health outcomes Community-level interventions Community randomized trials may be the best way to decide such public health policy issues as whether or not to use a community-wide anti-smoking program Community randomization may also be a good method for evaluating organ donation activities, particularly public awareness campaigns
Stampfer 1997 A methods paper discussing how and why large randomized clinical trials may not be feasible in evaluating the health effects of behavioral change, but rather how large-scale observational studies might provide useful information Methods paper for evaluating the health effects of behavioral change; Randomized trials versus large-scale observational studies Health effects Community-level interventions Due to limitations of using RCTs to evaluate health behavior change, observational studies continue to provide useful information Evaluations of organ donation activities face similar constraints as health behavior promotion activities; this paper describes options for study design that may help to improve the rigor of studies while not making them infeasible
Green 1995 Paper outlines an approach for the design and analysis of RCTs investigating community-based interventions for behavioral change aimed at health promotion Approach to community-matched trials. 11 matched pairs of communities, randomized at community level, number of communities determines power Outcome of smoking cessation Community-level smoking cessation trial Approach outlined helps improve the rigorousness and robustness of methods for evaluating the effects of community interventions Similar types of community interventions are employed in the organ donation world; this methodology and the suggested ways to improve study design could prove useful – particularly with regard to matching pairs of communities for intervention versus control
Guiffrida 1997 Reviewed randomized trials that tested the use of financial incentives to enhance patient compliance Review of other RCTs Compliance Patient-level intervention, financial incentives Financial incentives can improve patient compliance with medical advice As the use of financial incentives has been suggested as a means of increasing organ donation, this paper provides references for other financial incentive