We offer the following observations about the studies previously conducted to evaluate the NMEP.
CMS has sponsored numerous large and rigorously conducted evaluations of the NMEP. Studies include both quantitative and qualitative methods. Many of the individual studies appear to be of reasonable quality in terms of weighting responses to generalize the data to the larger beneficiary population and having adequate precision to assess access and use of NMEP components. Studies were predominantly of beneficiaries and placed little emphasis on proxy audiences.
P revious NMEP evaluations predominantly assessed (a) beneficiaries’ knowledge of Medicare and Part D; (b) beneficiaries’ awareness of the various NMEP components; (c) the appropriateness of information provided through various components to help beneficiaries understand their choices; and (d) beneficiaries’ use of and satisfaction with NMEP components.
P revious NMEP evaluations have lacked measures of whether NMEP and its components have led to change in the beneficiary population. These include whether beneficiaries (a) are more aware of their Medicare choices; (b) better understand and have improved knowledge of their choices; and (c) make informed health care choices at all and/or over time. There is little information on what people do with the information they access or are made aware of through NMEP or the knowledge they gain. It is also unclear how NMEP information is used to make health plan decisions.
Previous NMEP evaluations have placed little emphasis on evaluating the relative impact and cost-effectiveness of various NMEP activities.
The findings of previous NMEP evaluations suggest that beneficiary access to or awareness of NMEP resources and Medicare information appear to have increased over time; however, at levels lower than what the majority of the beneficiary population would need to successfully navigate through Medicare. Previous studies have not been longitudinal to be able to assess NMEP’s impact over time adequately, such as increase in knowledge. See Appendix A for an overview of prior studies.
The Program Assessment Rating Tool (PART) used by the Office of Management and Budget (OMB) to assess whether federal program and agencies are meeting their goals is a possible design element to consider for evaluating the NMEP. To date, only the Medicare program as a whole has been evaluated under the PART, with no specific mention of the NMEP.
In reviewing previous NMEP evaluations, we found that, since 2003, the focus of the NMEP has appeared to shift with the MMA and the introduction of the Medicare Part D program. Some refer to the NMEP as the Medicare & You campaign, possibly to increase “brand” recognition of the handbook. This changing focus of the NMEP suggests that it is an evolving program, which makes it responsive to the high-priority issues created by new legislation. It is consistent with CMS intentions for the program to be dynamic over time. However, this evolution toward a focus on prescription drug issues makes it challenging (particularly with declining agency resources) to focus on the broader set of Medicare topics that had historically been the focus of the NMEP, such as awareness of insurance option choices and managed care, that underpin the prescription drug program. This is particularly problematic because the MMA also introduced new managed care options and led to greater availability of options initially created under the BBA, such as private FFS plans. As a result, there is now “more to know” about Medicare, which also makes the issue of measuring increases in outcomes, such as awareness and understanding, more complex.