Evaluation of Selected Aspects of the National Medicare Education Program: Final Design Report. Challenges with Data on Outcomes and Effects


Just as NMEP activity costs change widely from year to year, the scope of NMEP activities also changes. If we are studying multiple years of the NMEP, the CEA estimates for one activity in 1 year may not be directly comparable to another year. To address this, one could collect as much information as possible on the subcomponents of each major activity and then determine which components are constant over time and which are not. Potential problems could be discussed in a limitations section.

The discussion above proposes the use of a Medicare beneficiary knowledge index (Uhrig et al., 2006 Uhrig et al., 2006) based on the MCBS as a common outcome measure for the CEA. The extent to which different NMEP activities are captured in the MCBS will dictate what can be measured and used in a CEA. If only three of five NMEP activities are captured, the other two will not be part of a CEA based on this measure. If the MCBS does not measure all activities, ASPE may wish to consider adding additional questions to the MCBS for long-term CEA studies.

Econometric identification of the effect of NMEP activities based on the MCBS knowledge index is also a concern. Analysis of the NMEP will rely on observational data from the MCBS. Econometric methods will be required to identify the effect of engaging in a particular NMEP activity on the outcome variable and to control for confounders, such as sociodemographics. Although econometric estimation should facilitate an unbiased measure of the relationship, direct causal inference is not possible without panel data. However, the MCBS sample design may permit limited panel data analysis. NMEP activities may also have a lagged effect, where participation (such as reading the Medicare & You handbook) affects the knowledge index in future years, not in the contemporaneous year. This could also be tested using panel data. Identification and lagged effects present challenges for further exploration but should not prevent implementation of the CEA.

View full report


"nmepreport.pdf" (pdf, 401.55Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®