Literature Review and Environmental Scan: Evaluation of Personal Health Records Pilots for Fee-for-Service Medicare Enrollees. Chapter 2. Methodology


To gather up-to-date information about PHR development and implementation for this comprehensive literature review we conducted a review of the existing published and unpublished literature, gray literature, and various official government documents. Based on our findings from the literature and through discussions with ASPE and CMS we identified an initial group of key informants who are involved with and otherwise knowledgeable about PHR development efforts, product design, standards, and usability/ utility. We completed discussions with sixteen key informants (see Appendix B). Finally NORC submitted a draft version of the literature review to a fourteen-member expert panel and incorporated their comments into the document.

A complete listing of sources consulted for the literature review is provided in Appendix C. NORC obtained recommendations for relevant materials and information sources by seeking initial guidance from the ASPE Task Order Monitor and key contacts at ASPE, CMS, AHRQ, and other federal agencies and foundations. We also conducted broad searches using the following online resources:

  1. Google
  2. Google Scholar– A more specialized search engine that focuses on peer-reviewed and other academic literature.
  3. Lexis Nexis– A search engine that primarily indexes proprietary content, including a range of public and trade periodicals.
  4. Academic Search Premier– A multi-disciplinary database of academic journal articles, drawing from over 3,700 peer-reviewed publications.
  5. MEDLINE– A computerized bibliographic retrieval system containing a comprehensive listing of articles in the scientific medical literature.
  6. HSRProj– A database providing access to ongoing grants and contracts in health services research available through the National Library of Medicine.
  7. HSTAT– Health Services/ Technology Assessment Text- A searchable collection of large, full-length text clinical practice guidelines, technology assessments, and health information.
  8. AHRQ National Resource Center for Health IT Knowledge Library– An online comprehensive search engine compiled by NORC and its partners, containing articles and information on topics such as evaluation, economics, and management of information systems.
  9. HIMSS– The online resources of the Health Information and Management Systems Society

The sixteen individuals who took part in the key informant interviews contributed a broadened perspective on how other organizations are currently sponsoring and delivering PHR functionality. Initial key informant interview participants were identified through the use of referrals from key Federal and other contacts. Subsequent interview subjects were identified by asking the initial participants to recommend other individuals who are known for their expertise on PHR development and implementation. Exhibit 1 summarizes information about the organizations represented in these discussions and specific topics addressed. A sample discussion guide is included in Appendix D.

NORC staff considered each of these three data sources in crafting the analysis presented in this document. Findings were synthesized and incorporated into the document through an iterative process in which NORC collected relevant data, and organized the information into broad categories. These categories were then mapped to the project’s research questions. A detailed outline of themes was developed and revised, ultimately taking the form of the chapters presented in this document. NORC staff outlined each chapter in detail, and revised chapter contents as the literature review and key informant discussions progressed.

Back to Top

EXHIBIT 1 Key Informant Discussion Participants and Topics

--------------->Key Informants

Sample of Key Informant Discussion Areas

X indicates that the key informant provided information in a discussion area.
Microsoft Corporation National Cancer Institute Markle Foundation Kaiser Permanente Life-Ledger AARP U. of Wisconsin-Madison Intuit Veterans Health Admin/ Consumers Union IHE AHIMA U. of North Carolina Whatcom HI Network Dell Group Health Incorporated
PHR Application or Platform
What initiatives are you involved with that are related to PHRs and standards development? X X X X X X X X X X X X X X X X
What have your project’s activities and lessons learned been to date in regards to developing user-centered personal health applications?       X     X                  
Usability and Utility
Was any usability testing done when rolling out or developing your PHR product? What type of testing was done? X     X X     X X             X
What do you consider to be important guidelines for user-centered design and for usability testing?   X   X     X X       X X      
How should usability be assessed for CMS’ PHR? Should usability guidelines differ for the Medicare population (elderly and disabled)? X X   X     X           X     X
Have there been usability "lessons learned" from your experience with your PHR that you think might be useful to CMS? What advice would you give to CMS? X X   X X   X X     X X X     X
What would you say are the key standards development activities related to PHRs with respect to security, privacy, and/or interoperability? X   X X X   X X X X X X   X X X
Are there any special considerations with respect to privacy and security when using a care-manager?     X X           X X X        
Are there any gaps in the current PHR standards development activities?     X       X       X X   X X X
Development of Best Practices
How do you see the business model of PHRs evolving over time? Are there other business models for PHRs that would serve as a good model for CMS?     X X         X              
What advice would you offer for CMS’ PHR–for now and the future?     X X   X   X           X X  
What are the advantages of using a claims-based model for a PHR? Other issues related to claims-based PHRs?     X     X X X       X X X   X
Consumer Perceptions
What do you think it will take for people to adopt and use PHRs?                 X X         X  
For what reasons do consumers decide to enroll in and maintain a personal health record?                 X X       X X  
Which features of the PHR do consumers find most helpful? Which do they like the best/ least? Are there functions that are not being used?       X X X     X X     X X X X
What kinds of effects do computer literacy and health literacy, and access to technology have on patient use of PHRs?                 X X         X X
Patient-Provider Interactions
How do PHRs affect the ways in which patients and providers interact?     X X                        
Do personal health records affect provider work flow?     X X                        
How have providers reacted to the PHR? Have provider attitudes to the PHR changed over time?     X X X       X         X   X
Issues Specific to Elderly and Underserved Populations
What do you see as the value of a PHR to Medicare beneficiaries?     X   X X       X     X X X  
Which features of the PHR are most valuable to the elderly?     X   X X       X     X X X  

View full report


"litreview.pdf" (pdf, 998.47Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®