Evaluation of Selected Aspects of the National Medicare Education Program: Final Design Report. 2.2.1 Outcomes Addressed


Previous studies have addressed each of the three goals of the NMEP awareness, understanding, and access/use/satisfactionwith the greatest number focusing on Medicare-related knowledge. Most studies devoted some attention to some of the common vulnerable subgroups. Overall, the findings of these studies suggest that, although the percentages of beneficiaries who have access to or are aware of NMEP resources and Medicare information appear to have increased over time, access and awareness remain lower than might be considered necessary for the majority of the beneficiary population to successfully navigate such a complex program. Several studies showed low to moderate levels of understanding of the Medicare program and related health insurance options, particularly in the beginning of the NMEP implementation period. In multiple studies (including those that used the MCBS), knowledge was higher among those beneficiaries who were exposed to the Medicare &You handbook. However, when the Medicare Current Beneficiary Survey (MCBS) was the data source used, the questions frequently had “true/false” response options, and those choices give respondents a high probability of guessing the correct answer. Although the MCBS was fielded on an ongoing basis over this period, we did not find external evidence that it was used to consistently monitor knowledge or the other NMEP goals.

An assessment of knowledge is frequently used in the communication literature as an intermediate outcome, but it has recognizable limitations because of the potential weak link between knowledge and actual behavior. In the studies we reviewed, we found little information on what people did with the information they had access to or were aware of , or the new knowledge they gained; for example, limited data were available on how the information was used to make health plan decisions. We view this as a research gap that remains to be filled. Similarly, we also did not see evidence of rigorous intervention and experimental design research, especially with respect to potential ways to educate beneficiaries about NMEP issues and whether changes are made as a result of such interventions. Longitudinal studies were also generally lacking, and they are important for examining impact over time, including decision quality. The MCBS may be one vehicle for this activity.

We found a fair amount of data assessing beneficiary satisfaction with the information available to them. In these studies, beneficiaries generally reported that they were satisfied with information available to them. Lower satisfaction levels were evident among some vulnerable subgroups: these subgroups also tended to use the NMEP information less frequently. Given the ceiling effects (i.e., high scores) that often occur with measuring satisfaction, it may be prudent to de-emphasize this variable.

Little emphasis has been placed on evaluating the relative impact and cost-effectiveness of various NMEP activities in relation to one another. The research has also not emphasized the effect of changing, enhancing, or otherwise modifying interventions to determine if these activities could improve beneficiary access or understanding of the information. Particularly given limited financial resources, more activities in this area may be warranted.

View full report


"nmepreport.pdf" (pdf, 401.55Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®