Evaluation of Selected Aspects of the National Medicare Education Program: Final Design Report. 4.3 Study Design Option 3: Prospective Cohort Study

A prospective cohort study is the most in-depth design option available for evaluating the NMEP, and this design has several advantages over other study options. First, the study will identify how key influences on Medicare decision making change over time. By following beneficiaries through the enrollment and plan decision processes, we can examine not only how factors such as values, NMEP materials, and current health care access influence Medicare plan decisions, but also how reliance on these factors increases or decreases over time. This information is key to understanding how beneficiaries arrive at their plan decisions and to determining the role NMEP components play in the decision process. To date, no evaluation of NMEP has examined beneficiary values or any trends in use of NMEP activities throughout enrollment.

Second, the study will identify whether and how multiple NMEP components work together to impact Medicare plan decision making. Currently, almost all NMEP evaluations have been cross-sectional studies that, at best, can identify associations between NMEP components and increased knowledge or plan satisfaction. Conversely, the prospective cohort design will examine how different NMEP components interact and what role each component plays in a beneficiary’s plan decision. This understanding will be crucial to recognizing when and how NMEP activities actually promote IDM among beneficiaries.

Finally, and most importantly, the design allows us to understand how and under what circumstances beneficiaries make active plan decisions. By following beneficiaries throughout enrollment, we can explore the thought processes used to make plan decisions, identify the most influential factors in those decisions, and determine to what extent IDM takes place. These insights are key to promoting IDM because we gain a better understanding of how NMEP activities can influence such decision-making processes. No evaluation to date has examined the decision-making process within the scope of NMEP activities.

The major advantage of this prospective cohort study is the ability to evaluate how Medicare beneficiaries’ enrollment status, plan satisfaction, and plan decisions change over time. By following beneficiaries from pre-enrollment (age 64) through the processes of initial enrollment, change-of-plan decisions, and, if applicable, disenrollment, we can identify the preferences, information, information sources, and experiences that drive those decisions.

Key research questions for evaluating the NMEP will include the following:

1. How do beneficiary preferences (such as plan cost, physician access and selection, specialist access, and continuity of primary care doctor) change over time? How do these values influence plan decisions?

2. How does participant access to and reliance on NMEP information sources (such as the Medicare & You handbook, www.Medicare.gov, 1-800-MEDICARE, and SHIPs) change over time? To what extent are these sources perceived as important in helping beneficiaries make decisions?

3. How does participant access to outside, non-NMEP information sources (such as employer human resources departments, private health insurance companies, and nonprofit groups like AARP) change over time? To what extent are these sources perceived as important in helping beneficiaries make decisions?

4. What do beneficiaries know about health insurance options and the Medicare program before enrollment? What experience do they have with selecting benefits and options? What experience do they have with the benefits and options offered by Medicare?

5. How does reliance on spouses, adult children, and other proxy decision makers change over time? What is the nature and extent of these individuals’ participation in making plan decisions?

6. How does satisfaction with access to information and the Medicare plan decision-making process change over time? Does satisfaction diminish over time following a plan decision? If so, when and how do beneficiaries act on this diminishing satisfaction?

Ultimately, we can use the study to identify which factors such as specific NMEP resourcesare catalysts for enrollment change and which are catalysts for staying enrolled in one’s current plan. More importantly, the study can help us understand beneficiary decision making about Medicare and how values, NMEP resources, and outside information influence those decisions, including fit between expressed preferences and actual choices.

Because understanding beneficiaries’ decision-making processes is a primary objective for this study, we propose using a theoretical foundation that helps explain IDM the Elaboration Likelihood Model (ELM) (Petty and Cacioppo, 1986Petty and Cacioppo, 1986). The ELM suggests that individuals can use two different cognitive processes to make decisions: a central processing route and a peripheral processing route. Individuals who use central processing try to evaluate information critically and exhaustively, consider the truthfulness and reliability of different arguments, and make a rational decision based on these considerations. This aligns closely with IDM, which entails a clear understanding of the issue and options available and a decision in line with one’s preferences and values (Briss et al., 2004Briss et al., 2004). Conversely, individuals who use peripheral processing make a decision that is not based on critical evaluation. Instead, individuals who use a peripheral decision-making process rely on peripheral cuessuch as likeability of an information source or perceived valueto guide their decision.

ELM also suggests that two constructs encourage central processing motivation and capacity. Motivation refers to an individual’s incentive to be engaged and involved in an issue (i.e., the perceived importance of selecting a Medicare health plan). Capacity refers to an individual’s ability to understand and assess information related to the issue (i.e., the ability to interpret a Medicare plan comparison chart). When individuals are both engaged in an issue and capable of assessing relevant information, they are more likely to use a central processing route and make an informed decision.

The ELM theory has important implications for understanding how Medicare beneficiaries make decisions about enrollment and health care plans. First, ELM can help identify beneficiaries who participate in IDM. Because individuals who use central processing are most likely to evaluate information critically, these same individuals are likely to make informed decisions that are consistent with their values. Second, ELM can help us identify NMEP components that are associated with and promote IDM. For example, the study may find that individuals who review www.Medicare.gov’s plan comparison information are more likely to be motivated to make, and capable of making, enrollment decisions.

Several sample selection and study duration options are available for the prospective cohort evaluation. We recommend that potential participants be randomly selected from the Social Security Administration files and the Medicare enrollment database (to ensure some representation from those eligible for Medicare due to disability) and that participants be enrolled in the study at least 1 year prior to Medicare enrollment (approximately age 64). This should ensure geographic and demographic representation across key groups and provide adequate time to collect data in advance of any Medicare enrollment decisions.

Ideally, the entire cohort will be selected at one time rather than recruiting participants on a rolling basis each year. This will shorten the overall time frame for the study and ensure that participants experience similar enrollment environments (e.g., cost of private health insurance at time of initial enrollment). To account for attrition (e.g., loss to follow-up, death), we recommend oversampling the initial cohort to ensure both that adequate data exist to answer the proposed research questions and that all participants are representative of the initial cohort.

To examine beneficiary decision-making processes adequately, we recommend the cohort study last several years, a decade or longer if possible, to ensure that beneficiaries have ample opportunity to make several plan decisions, including disenrollment. The major objectives of the study are to understand how beneficiary values and decision making change over time, and to achieve these objectives, the evaluation must extend long enough for numerous plan decisions to be made. The sample size would need to be driven by power calculations, stratification options, and funding availability.

Data for the prospective cohort study will be collected at regular intervals and will include both quantitative and qualitative approaches. Two instruments could be used for primary data collection a more quantitative self-administered, mail-returned survey and a one-on-one telephone discussion guide to collect more qualitative information. The survey will be the most frequently used instrument and will be administered at predetermined intervals.

Telephone discussions of a small subsample will occur less frequently and could be based on key events (i.e., disenrollment), providing more in-depth information to help interpret beneficiaries’ survey responses. This qualitative data will help elaborate on quantitative responses and will help determine whether beneficiaries participated in IDM. In addition to regular interviews, the study team will also interview beneficiaries when certain event triggers occur (e.g., beneficiary indicates on survey that he/she has changed Medicare plans).

We suggest a proposed data collection schedule that involves administering surveys every 6 or 12 months (depending on resources) and a few dozen telephone discussions each year. Because the study seeks to understand beneficiaries’ future intentions, we propose that individuals who disenroll from a certain plan continue to be surveyed as well. This data collection timeline will allow for adequate measurement of all key variables but should limit the burden on participants and minimize data collection costs, especially for telephone interviewing.

The cohort study will measure multiple outcomes and variables that depict beneficiary decision-making processes and how these processes change over time. The variables are broken down below into two categories outcome measures and independent measures.

Outcome Measures . For the cohort study, outcome measures include knowledge, satisfaction, final enrollment/disenrollment decisions, and decision-making processes. Because these outcomes are likely to change over time, we intend to measure them at several decision pointsinitial enrollment (baseline), change-of-plan decisions, and disenrollmentas well as at regular follow-up intervals. Table 4-2 describes the outcome measures in more detail.

Independent Measures and Other Variables . The study’s independent variables provide insight into what influences beneficiary knowledge, satisfaction, plan decisions, and decision-making processes. As with the outcome measures, independent variables will be measured at decision points as well as at regular follow-up intervals. Table 4-3 describes the independent variables in more detail.

Although the prospective cohort study design is one of the most rigorous and appropriate designs for evaluating NMEP activities, it has several challenges. The five primary obstacles for the study include attrition, limited recall of NMEP activities, diminished recall of the decision process, limited decision points during the study, and difficulty collecting data from special populations. Lack of stability in preferences and reports of preferences may be another challenge, given that people construct preferences as they undertake the decision-making process. A description of each challengeas well as the team’s proposed solutionsis detailed below.

Attrition. During the course of the study, a proportion of beneficiaries will likely be lost to attrition. Some of these may simply be lost to follow-up (e.g., change of address, nonresponse), but others may drop out because of death, institutionalization, or an incapacitating illness. Although the team will make every attempt to locate and collect data from participants, the team also recommends oversampling the initial population so that an adequate number of participants remains in the study even after attrition. Alternatively, if resources do not exist to oversample the initial cohort, the study could recruit additional beneficiaries as participants drop out. These replacements would need to be demographically similar to the dropped beneficiaries. However, this approach has several drawbacks, including lack of initial data for the replacement participants and loss of replacement participants to attrition.

Limited Recall of NMEP. Previous studies have demonstrated that beneficiaries may not accurately recall receiving or accessing NMEP resources, which may make it difficult for the study to distinguish between participants who did not access these resources and participants who simply do not recall accessing these resources. For example, approximately 20% to 30% of beneficiaries do not recall receiving the Medicare & You handbook even though the handbook is regularly mailed to all enrolled individuals (Bann et al., 2004Bann et al., 2004; Brant et al., 2001Brant et al., 2001). Telephone interviewers may help beneficiaries recall exposure to NMEP resources by using probing questions and describing the resource in considerable detail.

Diminished Recall of the Decision Process. Many beneficiaries may have difficulty recalling their decision-making process to enroll in a Medicare health plan, and the more time that elapses between the decision and data collection, the more difficult recall will be. To address this challenge, the study team has recommended frequent data collection intervals to minimize recall problems. Moreover, the team will use the survey data to flag decision points (e.g., decision to enroll in a new Medicare plan) and will follow up within 1 week to conduct a telephone interview. This approach increases the likelihood that beneficiaries will recall their decision process and rationale and that measurement of these variables will be accurate.

Limited Decision Points. Previous studies suggest that only about 10% to 12% of beneficiaries voluntarily change plans or completely disenroll from Medicare in a given year (Mobley et al., 2005Mobley et al., 2005; Lied et al., 2003Lied et al., 2003). These low change and disenrollment rates mean thataside from initial enrollmentonly a limited number of decision points will occur during the course of the study, limiting the opportunities to probe beneficiaries about their decision to change plans or stop using Medicare. While the study also is designed to explore beneficiaries’ decisions to remain in their current plan, such decisions are likely to be less formal and less explicit than the decisions to switch plans. To address this issue, the study team has recommended a multiyear study duration to maximize the number of disenrollment and change-of-plan decisions. The team has also recommended beginning the study prior to initial Medicare enrollment, which will allow the team to examine beneficiaries’ decision-making processes at that stage.