Evaluation of Selected Aspects of the National Medicare Education Program: Final Design Report. Appendix A


Summary of Previous NMEP Evaluations

Table A-1 summarizes previous NMEP evaluations (including basic data design, data source and population studied, sample sizes, and study period) first presenting the approaches used in CMS-sponsored analyses, followed by methods used in other government studies, and, finally, the methods used in studies conducted by nongovernmental organizations.




Sample Size

Study Time Period

Table A-1. Previous NMEP Evaluations

Centers for Medicare & Medicaid Services

Aduss, Grass, and Dahlberg, 2003

Video monitoring service Newspaper clipping service

N/A N/A 2001; 2002
Anderson, McCormack, Berkman et al., 2000 In-person survey (MCBS 1998) National probability sample of current Medicare beneficiaries who responded to the 1998 MCBS survey (rounds 22 24) Varies by question. No weighted range is 648,650 to 30,105,201 1998
Bann et al., 2000 Measurement testing National probability sample of current Medicare beneficiaries who responded to the 1995, 1996, 1997, or 1998 MCBS survey Varies by survey year and knowledge question. MCBS samples are usually 14,000+ individuals 1995 1998
Barth, Carlson, and Piacitelli, 2001

Telephone survey (NMEP Community Monitoring Survey)

In-depth interviews

Media monitoring

Medicare beneficiaries

Information intermediaries

Beneficiaries: 2,986 2000; 2001
Bearing Point, Sutton Group, Ketchum Public Relations, 2003

Focus groups

Telephone interviews (semi-structured)

Participant observation


Disabled Medicare beneficiaries (<65)

Caregivers of Medicare beneficiaries

Medicare beneficiaries (65+) who knew about prescription discount cards and comparison shop

Medicare beneficiaries (65+) who were unaware of prescription discount cards and don’t comparison shop

Key informants

Not specified Not specified
Gerteis and Ward, 2003

Media monitoring

Telephone surveys

Beneficiary diaries

In-depth interviews

Focus groups

Participant observation

Key informant interviews

Sample of Medicare beneficiaries who responded to NMEP Community Survey

Sample of hard-to-reach Medicare beneficiaries

Sample of key informants and information intermediaries

Community survey: 4,628 to 7,732

Beneficiary diaries: 32

Beneficiary interviews: 145

Beneficiary observation activities: 15

Key informant focus groups: 36

Key informant interviews: 89

1998 2003
Goldstein et al., 2001 Review of existing studies

Medicare beneficiaries

Upcoming beneficiaries


Health professionals

Existing call and Web data

Varies by study

Range is 200 to 16,693

Grad and Hassol, 2001

Telephone surveys

Focus groups

Randomly selected beneficiaries

Randomly selected M+C involuntarily disenrolled beneficiaries

Beneficiaries: 2,036

M+C beneficiaries: 2,048

Greenwald et al., 2006 Review of existing studies Current and new Medicare beneficiaries Varies by study 2001
Harris-Kojetin et al., 2001 Focus groups

Current Medicare beneficiaries in Kansas City, MO, area who participated in a previous survey

Dually Medicare/Medicaid eligible individuals and individuals eligible for Medicare because of disability

Beneficiaries: 56  
James, Neuman, and Kitchman Strollo, 2006 Focus groups Representatives from the SHIP agencies in 13 states Not specified 2006
Levesque and Cummins, 2004


Mailed surveys

National random sample of upcoming enrollees from CMS’ Enrollment File

Upcoming enrollees: 855

Intervention group: 179

Control group: 676

Not specified

McCormack, Anderson, Daugherty et al., 2001

McCormac k, Anderson, Kuo et al., 2001

McCormack, Anderson, Uhrig et al., 2001

McCormack, Garfinkel, Hibbard, Kilpatrick et al., 2001

McCormack, Garfinkel, Hibbard, Norton et al., 2001

McCormack and Uhrig, 2003

Mailed survey

Focus groups

National random sample of Medicare beneficiaries from 1998 enrollment database

Total sample: 3,738

Treatment (both groups): 2,563

Control group: 1,175

2000; 2001
McCormack, Garfinkel, Hibbard, Keller et al., 2002


Telephone survey

New and experienced Medicare beneficiaries in Kansas City area, aged 65 or older (or nearing 65 th birthday) Beneficiaries: 2,107 Not specified
CMS, 1999

Focus groups

In-person survey (MCBS)

Medicare beneficiaries

Organizations that work directly with Medicare beneficiaries

Not specified 1996 1997
CMS, 1999

Focus groups


Medicare beneficiaries (not specified) Not specified 1999
Pacific Consulting Group, 2006 Telephone survey

Callers assisted by customer service representative (English-speaking)

Callers assisted by customer service representative (Spanish-speaking)

Callers assisted by interactive voice system (English-speaking)

Customer service callers (English): 18,008

Customer service callers (Spanish): 1,125

Interactive voice system callers: 4,500


Sing et al., 2001

Sing and Stevens, 2005

Mailed survey

New enrollees those who joined an HMO during sample intake period

Switchers enrolled in one Medicare HMO and switched to a different HMO

FFS enrollees

Beneficiaries: 3,125 1999 2001
Sofaer et al., 2001 Focus groups Individuals aged 65 or over who had helped another beneficiary Not specified 2001
Uhrig et al., 2006 In-person survey (MCBS) Medicare beneficiaries who were administered Medicare Knowledge supplement during Round 36 of MCBS from May to August 2003 Beneficiaries: 2,497 2003
Government Accountability Office
U.S. Government Accountability Office, May 2006

Audit of CMS publications

Mystery shopping calls

Web site usability testing


Publications: 70

Calls: 500

2005 2006
U.S. Government Accountability Office, 2001

Audit of NMEP materials and telephone records

Audit of CMS-contracted research reports

In-depth interviews

Representatives from beneficiary advocacy groups (AARP, Medicare Rights Center, and Center for Medicare Education)

Health Care Plan Associations (American Association of Health Plans and Health Insurance Association of America)

Officials in two HCFA regional offices

Not specified 1998; 2000
National Institutes of Health
Winter et al., 2006 Internet survey Medicare-eligible population (aged 65 or older, or those who will be eligible within 2 years) Seniors: 4,738 2006
Kaiser Family Foundation
Kaiser Family Foundation, 2006 Structured interviews Not specified Beneficiaries: 35 2005; 2006
Kaiser Family Foundation, 2006 Public opinion polls Seniors 65 or older Varies by poll 2006
Smith et al., 2005 Focus group Not specified Medicaid officials: 12 2005
Kaiser Family Foundation, Harvard School of Public Health, 2005 Public opinion poll Nationally representative sample of U.S. adults aged 65 or older Seniors: 802 2005
Commonwealth Fund
Leatherman and McCarthy, 2005 Review of existing studies

Current Medicare beneficiaries

Medicare beneficiaries enrolled in specific programs (e.g., HMO, FFS)

Hospitalized Medicare beneficiaries

Community-dwelling elderly adults

Varies by study 1996 2004
Robert Wood Johnson Foundation
Gold et al., 2001 Telephone survey Current Medicare beneficiaries selected through survey algorithm Beneficiaries: 6,620 2000
California Health Care Foundation
California Health Care Foundation, 1999 Telephone survey Random sample of California seniors (65+) covered under Medicare Seniors: 755 1999
Cihak, 2006 Telephone survey Random sample of California adults who recently turned 65 or were about to turn 65 Seniors: 288 2006
Cihak, 2006

Telephone survey

Focus groups

Random sample of California adults aged 64

Purposive sample of Medicare intermediaries

Upcoming beneficiaries: 417

Intermediaries: 38

MedPAC, 2006

Telephone survey

Focus groups

Structured interview


Family members of beneficiaries

SHIP counselors and coordinators

Survey: 1,411

Focus groups 72

Interviews: 30

American Association of Retired Persons
Hibbard and Jewett, 1998 Telephone survey Medicare beneficiaries with Part A and Part B aged 65 to 80 living independently in their community Beneficiaries: 1,673 1998

View full report


"nmepreport.pdf" (pdf, 401.55Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®