Evaluation of Selected Aspects of the National Medicare Education Program: Final Design Report. 4.3.7 Potential Challenges

01/10/2008

Although the prospective cohort study design is one of the most rigorous and appropriate designs for evaluating NMEP activities, it has several challenges. The five primary obstacles for the study include attrition, limited recall of NMEP activities, diminished recall of the decision process, limited decision points during the study, and difficulty collecting data from special populations. Lack of stability in preferences and reports of preferences may be another challenge, given that people construct preferences as they undertake the decision-making process. A description of each challengeas well as the team’s proposed solutionsis detailed below.

Attrition. During the course of the study, a proportion of beneficiaries will likely be lost to attrition. Some of these may simply be lost to follow-up (e.g., change of address, nonresponse), but others may drop out because of death, institutionalization, or an incapacitating illness. Although the team will make every attempt to locate and collect data from participants, the team also recommends oversampling the initial population so that an adequate number of participants remains in the study even after attrition. Alternatively, if resources do not exist to oversample the initial cohort, the study could recruit additional beneficiaries as participants drop out. These replacements would need to be demographically similar to the dropped beneficiaries. However, this approach has several drawbacks, including lack of initial data for the replacement participants and loss of replacement participants to attrition.

Limited Recall of NMEP. Previous studies have demonstrated that beneficiaries may not accurately recall receiving or accessing NMEP resources, which may make it difficult for the study to distinguish between participants who did not access these resources and participants who simply do not recall accessing these resources. For example, approximately 20% to 30% of beneficiaries do not recall receiving the Medicare & You handbook even though the handbook is regularly mailed to all enrolled individuals (Bann et al., 2004Bann et al., 2004; Brant et al., 2001Brant et al., 2001). Telephone interviewers may help beneficiaries recall exposure to NMEP resources by using probing questions and describing the resource in considerable detail.

Measure

Description

Instrument

Table 4-2. Prospective Cohort Outcome Measures

Initial Enrollment

Enrollment Decision

Choice of plan for initial Medicare enrollment (e.g., FFS, HMO)

Survey

Satisfaction Decision

Satisfaction with initial enrollment decision (i.e., how satisfied is beneficiary with the plan he/she chose?)

Survey

Discussion Guide

Satisfaction Process

Satisfaction with initial enrollment process (i.e., how satisfied is beneficiary with available information and plan options?)

Survey

Discussion Guide

Demonstrated Knowledge

Actual, rather than perceived, knowledge of Medicare and selected plan option

Survey

Decision-Making Process

Central or peripheral decision process used to arrive at decision (based on motivation, capacity, and rationale)

Survey

Discussion Guide

Change-of-Plan Decision

Enrollment Decision

Choice of plan (e.g., FFS, HMO)

Survey

Satisfaction Decision

Satisfaction with plan decision (i.e., how satisfied is beneficiary with the plan he/she chose?)

Survey

Discussion Guide

Satisfaction Process

Satisfaction with plan change process (i.e., how satisfied is beneficiary with available information, plan options, and ease of switching plans?)

Survey

Discussion Guide

Demonstrated Knowledge

Actual, rather than perceived, knowledge of Medicare and new plan option

Survey

Decision-Making Process

Central or peripheral decision process used to arrive at decision (based on motivation, capacity, and rationale)

Survey

Discussion Guide

Disenrollment

Satisfaction Decision

Satisfaction with disenrollment decision (i.e., how satisfied is beneficiary with choice to ultimately disenroll?)

Survey

Discussion Guide

Satisfaction Process

Satisfaction with disenrollment decision (i.e., how satisfied is beneficiary with available information and ease of disenrollment?)

Survey

Discussion Guide

Enrollment Intentions

Intention to enroll in other health insurance program or intention to re-enroll in Medicare in future (i.e., does beneficiary intend to enroll in a private health insurance plan? Does beneficiary intend to re-enroll in Medicare in the future?)

Survey

Decision-Making Process

Central or peripheral decision process used to arrive at decision (based on motivation, capacity, and rationale)

Survey

Discussion Guide

Regular Follow-Up (No Plan Change)

Satisfaction Decision

Current satisfaction with enrollment decision (i.e., how satisfied is beneficiary now with enrollment decision?)

Survey

Discussion Guide

Demonstrated Knowledge

Actual, rather than perceived, knowledge of Medicare and selected plan

Survey

Enrollment Intentions

Intention to remain in current plan, switch to a new Medicare plan, or disenroll from Medicare in the future

Survey

Decision-Making Process

Central or peripheral decision process used to arrive at decision to remain in current plan (based on motivation, capacity, and rationale)

Survey

Discussion Guide

Decision Point (Initial Enrollment, Change-of-Plan, Disenrollment)

Decision Rationale

Reason for enrolling, changing plans, or disenrolling and reason for selecting plan option

Survey

Discussion Guide

Beneficiary Values

Preferences and values considered when making decision (e.g., cost, physician access, quality of care)

Discussion Guide

Preference Priority

Priority and importance of articulated values

Discussion Guide

Decision Discussion

Discussion of decision options with spouse, children, or other proxies

Survey

Discussion Guide

Proxy Influence

Perceived influence of spouse, children, or other proxies on final decision

Survey

Discussion Guide

Health Services Access

Recent use of health services

Survey

Awareness Primary NMEP Resources

Beneficiary awareness of primary NMEP activities (e.g., Medicare & You handbook, www.Medicare.gov, 1-800-MEDICARE, Consumer Assessment of Healthcare Providers and Services [CAHPS])

Survey

Awareness Other NMEP Resources

Beneficiary awareness of other NMEP activities (i.e., SHIPs, REACH activities, HORIZONS activities, plan comparison database, physician directories)

Survey

Use NMEP Resources

Use and extent of use of NMEP resources

Survey

Discussion Guide

Trust NMEP Resources

Perceived accuracy of NMEP resources

Survey

Discussion Guide

Awareness Outside Resources

Beneficiary awareness of non-NMEP resources (i.e., employer human resource departments, private health insurance companies, nonprofit organizations)

Survey

Use Outside Resources

Use and extent of use of non-NMEP resources

Survey

Discussion Guide

Trust Outside Resources

Perceived accuracy of non-NMEP resources

Survey

Discussion Guide

Regular Follow-Up (No Plan Change)

Awareness Primary NMEP Resources

Beneficiary awareness of primary NMEP activities (e.g., Medicare & You handbook, www.Medicare.gov, 1-800-MEDICARE, CAHPS)

Survey

Awareness Other NMEP Resources

Beneficiary awareness of other NMEP activities (e.g., SHIPs, REACH activities, HORIZONS activities, plan comparison database, physician directories)

Survey

Use NMEP Resources

Use and extent of use of NMEP resources

Survey

Discussion Guide

Trust NMEP Resources

Perceived accuracy of NMEP resources

Survey

Discussion Guide

Awareness Outside Resources

Beneficiary awareness of non-NMEP resources (e.g., employer human resource departments, private health insurance companies, nonprofit organizations)

Survey

Use Outside Resources

Use and extent of use of non-NMEP resources

Survey

Discussion Guide

Trust Outside Resources

Perceived accuracy of non-NMEP resources

Survey

Discussion Guide

Diminished Recall of the Decision Process. Many beneficiaries may have difficulty recalling their decision-making process to enroll in a Medicare health plan, and the more time that elapses between the decision and data collection, the more difficult recall will be. To address this challenge, the study team has recommended frequent data collection intervals to minimize recall problems. Moreover, the team will use the survey data to flag decision points (e.g., decision to enroll in a new Medicare plan) and will follow up within 1 week to conduct a telephone interview. This approach increases the likelihood that beneficiaries will recall their decision process and rationale and that measurement of these variables will be accurate.

Limited Decision Points. Previous studies suggest that only about 10% to 12% of beneficiaries voluntarily change plans or completely disenroll from Medicare in a given year (Mobley et al., 2005Mobley et al., 2005; Lied et al., 2003Lied et al., 2003). These low change and disenrollment rates mean thataside from initial enrollmentonly a limited number of decision points will occur during the course of the study, limiting the opportunities to probe beneficiaries about their decision to change plans or stop using Medicare. While the study also is designed to explore beneficiaries’ decisions to remain in their current plan, such decisions are likely to be less formal and less explicit than the decisions to switch plans. To address this issue, the study team has recommended a multiyear study duration to maximize the number of disenrollment and change-of-plan decisions. The team has also recommended beginning the study prior to initial Medicare enrollment, which will allow the team to examine beneficiaries’ decision-making processes at that stage.

View full report

Preview
Download

"nmepreport.pdf" (pdf, 401.55Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®