Evaluation of the Personal Health Record Pilot for Medicare Fee-For Service Enrollees from South Carolina

Contract No: HHSP23320045020XI

Prepared for:

Ms. Suzie Burke-Bebee
U.S. Department of Health and Human Services
Office of the Secretary
Assistant Secretary for Planning and Evaluation

Prepared by:

NORC at the University of Chicago
4350 East West Highway, Suite 800
Bethesda, MD 20814
301-634-9300 
301-634-9301 (fax)

"

Acknowledgements

The National Opinion Research Center (NORC) presents this report, Evaluation of the Personal Health Record pilot for Fee for Service Medicare Enrollees.  This study was funded by the Assistant Secretary for Planning and Evaluation (ASPE) and with the Center for Medicare and Medicaid Services (CMS) of the Department of Health and Human Services.  The NORC project team includes Prashila Dullabh, Dan Gaylin, Randy Horton, Elizabeth Babalola, Alison Muckle, Casey Hogle, Tom Briggs, June Eichner, Karen

NORC gratefully acknowledges the contributions of our Project Officer, Suzie Burke-Bebee, who provided thoughtful input and guidance throughout the project.  We would also like to thank Lorraine Doo, Elizabeth Holland, Chrislyn Gayhead, and Rosali Topper at CMS for their continued assistance.  Additionally, NORC appreciates the ongoing guidance of the Personal Health Record (PHR) Expert Panel, comprised of twelve prominent leaders with expertise in the area of usability design, evaluation of PHR applications, PHR standards, and implementation. The members of this panel provided guidance throughout the evaluation on the approach to evaluation activities, instrument development, results interpretation and current work in PHRs.  A list of the Expert Panel members is included in Appendix 1. Finally, we would also like to acknowledge the contributions made by Patrician Flatley Brennan, who served as an expert consultant to NORC on this project.

This study was approved by the NORC Institutional Review Board under protocol number 6488-02-62.  For more information, please contact Prashila Dullabh, the Project Manager via phone, at (301) 634-9418 or e-mail at dullabh-prashila@norc.org.  

Executive Summary

The Office of the Assistant Secretary for Planning and Evaluation (ASPE), in close collaboration with the Centers for Medicare and Medicaid Services (CMS), has contracted with the National Opinion Research Center (NORC) at the University of Chicago to conduct an evaluation of a pilot demonstration project to introduce Personal Health Records (PHRs) to Medicare fee-for-service (FFS) beneficiaries.  The pilot study represents one component of a diverse set of CMS Health Information Technology (Health IT) initiatives. This pilot is being implemented in the South Carolina service area by Quality Software Services Incorporated (QSSI), an information technology (IT) solutions contractor.

This review is intended to contribute to the overall development of PHRs by providing a current-state, panoramic snapshot of many key aspects of the PHR field.  The report will inform policy efforts to identify best practices for developing PHR features and standards that will encourage PHR adoption by consumers, especially Medicare FFS beneficiaries. 

Background

To date, little work has been conducted to examine what, if any, assistance may be needed to help elderly and disabled populations use PHRs.  Due to the potential of PHR technology to provide benefits for patients such as better patient-provider communication, improved quality of care, reduction in unnecessary tests and medication errors, and improvements in overall health, additional insight is needed to identify PHR features and functions that will encourage PHR adoption by consumers. A 2003 online survey conducted by the Markle Foundation found that individuals with chronic illnesses and those caring for the elderly reported the highest need and most urgent interest in PHRs.^[1]  In a more recent  study, Kaebler et al. (2008) found that those with chronic conditions could potentially benefit the most from using PHRs.^[2]  However, there are many barriers to adoption of PHRs for this population, and little research is available on the usability and utility components of PHRs for beneficiaries.

CMS has been working to demonstrate the value of PHRs for Medicare beneficiaries since 2005. In 2007, CMS initiated two contracts regarding Medicare FFS and Managed Care beneficiaries’ use of PHRs. Findings from these projects will inform a third CMS pilot initiated in 2009 which will allow Medicare FFS beneficiaries from Arizona and Utah the choice of registering for one of four PHRs, each of which offers a variety of functions.

CMS contracted with QSSI to administer the pilot PHR, entitled “My Personal Health Record, South Carolina” or MyPHRSC.  The QSSI team utilized HealthTrio’s commercially available PHR and Palmetto GBA’s expertise with CMS claims data in South Carolina to provide a PHR that contained a beneficiary’s last two years of claims data.   Outreach activities began in February 2008 targeting 100,000 Medicare FFS beneficiaries located in South Carolina. Outreach events ranged from attendance at health fairs to presentations and small-group workshops at senior centers. QSSI pre-populated the PHR with basic demographic information and limited clinical information from the claims files through Palmetto GBA.

ASPE contracted with NORC to conduct an evaluation of the FFS pilot.  The ASPE and CMS goals for the project included: 1) identifying the key usability and utility components of PHRs; 2) identifying best practices related to the development and deployment of PHRs, including standards that could be used for development and use; and 3) exploring the key issues and perceptions related to PHR use and adoption.

This report is an overall synthesis of the findings from the PHR literature review and environmental scan that was completed early in the project as well as findings from the qualitative and quantitative evaluation activities performed later which focused on beneficiary experiences with My Personal Health Record South Carolina (MyPHRSC). The report will inform policy efforts about new PHR technology; best practices and standards used to develop PHR features and functions; and the implications the features and functions had on MyPHRSC adoption and use by Medicare FFS beneficiaries. 

Methods

NORC designed a largely qualitative evaluation that was conducted in 3 phases. Phase 1 of the project included a broad environmental scan including an extensive literature review and discussions with sixteen key informants, as well as background discussions with MyPHRSC contractors. 

The environmental scan provided the context and scope for phase 2 of the project, which focused on beneficiaries using the MyPHRSC tool.  Phase 2 included discussions with beneficiary MyPHRSC users and nonusers, laboratory observations of beneficiary MyPHRSC users, discussion with providers in South Carolina, and analysis of utilization statistics.  Phase 3 of the evaluation, the final report, details the background and methodology for all evaluation activities in this pilot, highlights key findings, summarizes key themes and provides conclusions and recommendations for future research.

For each evaluation activity, NORC worked in close collaboration with ASPE, CMS, Patricia Flatley Brennan, and the members of the PHR Expert Panel. 

Findings for each qualitative evaluation activity investigated beneficiary healthcare priorities, methods for managing health information, computer experiences, and perceived impact of using the PHR to assess beneficiary preferences. The key challenges and barriers as well as enablers for PHR use for this population were also identified.  Finally, analysis of MyPHRSC utilization data revealed beneficiary PHR usage patterns, frequency of use, preferences for PHR functions, and usability components of PHRs for this population. 

Results

Beneficiary Perceptions of MyPHRSC. Generally, participants felt that the PHR could be a very useful tool for them for obtaining health information and indicated that they viewed the PHR as a method for improving communication with their providers. Many were impressed with the thoroughness of the information included in MyPHRSC and commended the accessibility features, such as the ability to change the font size within the PHR.

The most popular features of MyPHRSC included the Wallet Card (a printable summary of pertinent health information like medications and emergency contact information), which beneficiaries viewed as a mechanism to share information with providers, to use in an emergency situation, or to carry during travel. The health record summary (an online view of comprehensive information about beneficiaries’ health and care over a 2 year period) and the claims history (an online summary of processed claims accessible from MyPHRSC) were also popular features of MyPHRSC.  

Health Information Management Practices. Beneficiaries viewed themselves as the primary owners of their health information, and most had established mechanisms for managing their own care.  Discussions with MyPHRSC users and non-users also revealed that beneficiaries had many methods for managing their care including storing paper files and keeping paper calendars.  Overall, beneficiaries considered the PHR a source of supplemental health information rather than a replacement for their current health records, and they used it as just one component of their greater information management strategy.

Experience with Computers. Most beneficiaries reported using computers regularly and most owned computers in their homes. However, many did not consider themselves computer savvy. 

MyPHRSC Usage Patterns. In examining the general trends of MyPHRSC usage over time, it is clear that beneficiaries’ use of the PHR drops off one or two months after they initially register. The majority of beneficiaries who used MyPHRSC logged in during only one calendar month of the pilot. Additionally, over one-quarter of registrants never logged in to MyPHRSC after receiving their user identification (ID) and password. Based on findings from the discussion groups with beneficiaries, interviews with non-users and observational studies, beneficiaries may never log in because they do not see enough value in the PHR for continued use, or because of difficulties remembering and using their log in information.   Additionally, the PHR may not fit into beneficiaries’ pre-established Internet “workflow”.  Follow up contact through e-mail messages or CMS general mailings might be helpful to encourage repeat use of MyPHRSC.  Such reminders could also be triggered when new information auto-populates the PHR or when a specified time-lag occurs in using the PHR.

Barriers to MyPHRSC Use. One of the key challenges that prevented users and nonusers from continuing PHR use was the lack of ease for logging in and the inability to remember user IDs and passwords.  Beneficiaries also reported having some difficulty manually entering their data into the PHR, particularly in entering medications on the Medications page. Additionally, the beneficiaries found medical jargon contained within the PHR difficult to understand, impeding their use and reuse of the PHR.

Lack of provider interest in the PHR was another barrier to beneficiaries’ use of the tool. On the whole, the discussion group conducted with providers revealed that they viewed patients’ use of PHRs and their sharing health information positively. However, providers also revealed that concerns about workflow disruptions, validity and reliability of information in the PHR, reimbursement issues and concerns about the limitations of claims data (versus clinical data) all acted as barriers to their acceptance and use of PHRs.

Illness/Condition Diagnosis. It was hypothesized that there would be greater use of MyPHRSC among registrants with the following conditions: high blood pressure, coronary artery disease, heart failure, diabetes, chronic obstructive lung disease, asthma, osteoarthritis and cataracts. However, analysis of the usage data showed that there were no consistent patterns of PHR use based on illness diagnoses. It is noted that this study period of one year and the sample size of actual users may not be sufficient to draw conclusions on linkages between chronic conditions and PHR use.

Impact of Outreach Events. On the whole, there was insufficient data to determine the effectiveness of outreach activities on beneficiary registration for MyPHRSC.  While several specific outreach events seemed to be linked to an uptake in beneficiary registrations on a particular day, there were substantially more outreach events that were not linked to any increase in registration for MyPHRSC. 

Conclusions and Recommendations.

Based on findings from the evaluation of the FFS PHR pilot, we offer a set of conclusions and recommendations that are intended to provide insight into future research related to Medicare beneficiaries and PHR usability and utility:

• Auto-population of data. Although beneficiaries did not agree on the maximum amount of data they would like access to nor did they identify a standard dataset that would be most useful to view in a PHR, the overwhelming consensus was to include as much information as possible in as clear a manner as possible. 
• Simplification of login process. A significant barrier to beneficiary use of PHRs was the lack of a single easy, successful login. Simplifying the login process by allowing user-generated login names and password would serve to address this issue.  Additionally, passwords were temporary and lasted only 90 days before requiring reset, and beneficiaries found it difficult to generate and remember new passwords.
• Inaccurate or missing data. Future claims-based PHR tools should be developed considering the claims data process (submitted, reviewed, accepted, reimbursed) to ensure that PHR data is accurate (reliable and valid), understandable by the user and provided in a timely manner.
• Technical support. In order to facilitate PHR ease-of-use, strong technical support should be easily accessible initially and on an as-need basis for as long as the PHR is available to the user.
• Recommendations for additional functions. Inclusion of other functions and features such as an auto-populated medication history page, laboratory results, clinical decision support (e.g. health screening alerts), enhanced health topic search, and secure messaging (email) with providers would enhance PHR utility for beneficiaries.
• More comprehensive medical record. Beneficiaries want as much relevant personal health information in their PHR as possible, which includes both claims data as well as clinical information from their providers. The coordination of different insurers (e.g. co-pays) was also relevant to the users.
• Chronic Illness Management. Functions such as tracking medical conditions, provider appointments and medication reminders (e.g. refills), and graphing of vital signs over time should be included to support beneficiary management of chronic illnesses. 
• Facilitation of patient-provider communication. Allowing beneficiaries the capability to share the PHR directly with their various providers whether electronically or in print would contribute to an increased coordination of care and serve to foster patient-provider communication.

NORC also identified several critical areas for future study of PHRs and Medicare beneficiaries.  In discussions and observations, beneficiaries demonstrated the desire to be able to manage all of their health information in a central location.  Further evaluations should explore how PHRs can be used to coordinate patient care across the continuum of care.  Additionally, future research investigating PHR utilization among beneficiaries will benefit from examining linkages between outreach activities and PHR use (and reuse), and from systematically studying the incidence of chronic conditions and PHR use.  An examination of PHR use by authorized representatives was beyond the scope of this evaluation but will be important for future studies as beneficiaries have reported this feature to be highly desired for inclusion in a PHR. Our findings have consistently cited that beneficiaries view provider access to the PHR as a central component in a PHR. Studies to determine which type of PHR (e.g. independent or tethered) provides the most user satisfaction will help explain PHR value for both providers and patients.  Finally, developing approaches to better integrate PHRs into beneficiary workflows as well as joint workflows between beneficiaries and providers (e.g. healthcare encounters) will be a key element in encouraging use of the system.

Background

Overview of PHRs.

For the purposes of this report, Personal Health Records, often known as PHRs, are defined based on the National Alliance for Health Information Technology (NAHIT) standard definition: ‘An electronic record of health-related information on an individual that conforms to nationally recognized interoperability standards and that can be drawn from multiple sources while being managed, shared, and controlled by the individual.’[3] PHR applications may differ in the nature of information which they contain, the features and functions they offer, their sources of information, the locations where information is stored, their technical approaches to security, and the designation of access control. Because of this diversity, no universal definition of what constitutes a PHR has yet been adopted. Today, commonly used definitions refer to characteristics of PHRs (e.g., an electronic application accessed and managed by consumers through which personal health information is maintained and shared in a secure, private, and confidential environment.

Exhibit 1.   Research Framework Questions

Key:   

X Primary   
O Supplementary

Pre Discussion Activities Discussions Observational Studies Structured Interviews: Contractors Structured Interviews: Vendors Analysis of Usage data Providers Beneficiaries Beneficiaries Usability             How effectively can stakeholders use the PHR to obtain the information of interest to them?   X X O X X What appearance, design, and visual treatment options would increase usability (including for Section 508 compliance)?   O X O X X What screen elements are confusing or difficult to use?   O X O X X Utility             What key information should the PHR contain? O X O X X X What are the key features the PHR should include? O X O X X X How accessible is the information in the PHR? O X O X X X What features contribute to user frustration? O X O X X X Issues / Concerns             How accurate/reliable is the system? X X O O O O What implications does the PHR use have on providers?       X O O Perceptions of Impact/Benefits             How do PHRs affect patient-provider interactions?       X X X How do PHRs affect patient self-care and self-management? O O O O X X How do PHRs affect patient outcomes? O O O O X X

Evaluation Activities.

The project was conducted in three phases. Phase 1 involved a broad review of existing available information on PHRs and MyPHRSC, including an environmental scan and literature review and background discussions with MyPHRSC contractors. Phase 2 of the evaluation included: 1) discussion groups with beneficiary users of MyPHRSC; 2) semi-structured interviews with nonusers of MyPHRSC; 3) observations of MyPHRSC users; 4) a discussion group with providers and 5) quantitative analysis of MyPHRSC usage data.  Phase 3 of the evaluation involved a synthesis of findings from all evaluation activities and the production of a final report.

Exhibit 2:   Phases of the Project and Evaluation Components

Exhibit 2 provides an overarching framework for the phase of the project and the evaluation approach that NORC undertook.

Phase 1: PHR Environmental Scan and Background information on MyPHRSC.

Environmental Scan and Literature Review.  In order to assess the current state of PHRs and gain an understanding of the key factors related to PHR use and adoption, NORC began the evaluation by conducting an extensive review of the existing published literature, gray literature, and various official government documents.  Additionally, NORC completed discussions with sixteen key informants who were involved with and otherwise knowledgeable about PHR development efforts, product design, PHR standards, and PHR usability and utility.  These individuals and organizations provided a broader view of how other organizations that were already sponsoring and/ or delivering PHR functionality viewed their own efforts, as well as those of Medicare. Subsequent participants were identified through the use of “snowballing” techniques, in which subjects were asked to recommend other individuals who were knowledgeable about PHR development and implementation.  A list of key informants is included as Appendix 3. 

Exhibit 3.   PHR Environmental Scan/Literature Review Methodology

Exhibit 3 outlines the process and methodology for the development of the NORC environmental scan and literature review.

In synthesizing lessons learned from the literature review and key informant discussions, we focused on issues in four key areas: (1) PHR definitions, attributes and models; (2) development of consumer oriented PHRs; (3) standards for PHRs; and (4) effects of PHRs on the U.S. health care system.  While the environmental scan focused broadly on these areas, NORC made a concerted effort to include available information on what was known about the use of PHRs in the Medicare population to inform subsequent evaluation activities.  Thus, findings from the literature review and environmental scan informed the design and structure of subsequent evaluation activities.  All other evaluation activities focused on Medicare beneficiaries’ experiences with MyPHRSC. 

Background Discussions with MyPHRSC Contractors.  In addition to the extensive review of the literature and environmental scan, NORC conducted sixty-minute semi-structured telephone interviews with each of three MyPHRSC contractors involved in implementing the pilot program, including: QSSI (prime contractor ), HealthTrio (the PHR vendor) and Palmetto GBA (claims data provider). We discussed general themes with all three contractors that included overall experiences with MyPHRSC, the claims data process, issues around security and other related concerns in administering a PHR for Medicare, and insights related to beneficiary experiences with MyPHRSC.  Discussion guides for the telephone conversations are included in Appendix 4.

For interviews with HealthTrio, the NORC team asked for information on the various functionalities of the PHR tools and about procedures for providing customer service to both the purchaser and the users.  For interviews with QSSI, the team asked about outreach activities, beneficiary experiences with MyPHRSC and customer service.   In addition, these interviews served as an ideal opportunity to collect more detailed accounts regarding the strategy or planning behind each organization's decision to participate in the PHR pilot and the extent to which that vision has been realized.  Table 1 below summarizes the discussion topics for PHR vendors and managers and identifies areas of overlap.

Table 1.     Discussion Topics for Interviews with PHR Contractors

	PHR Functions	Customer Service Procedure	Beneficiary Experiences with MyPHRSC	Staff Experience with the PHR Vendor Vendor	Strategy/ Plan for Pilot Participation	Data lifecycle	Standards	Security	Outreach	Challenges and Barriers
HealthTrio	X	X	X	X	X	X	X	X		X
QSSI	X	X	X	X	X	X	X	X	X	X
Palmetto GBA		X		X	X	X	X	X		X

Detailed notes were taken during telephone interviews with MyPHRSC contractors. Based on these notes, NORC created a single summary document which synthesized results from each discussion.  These interviews provided NORC with the necessary context to more effectively lead discussions with beneficiaries, and to allow a deeper exploration of key issues related to PHR development and organization from a vendor and contractor perspective. 

Phase 2: Beneficiary experiences with MyPHRSC.

Each qualitative evaluation activity explored the beneficiaries’ healthcare priorities, methods for managing health information, computer experiences, challenges, and perceived impact of using the PHR in order to assess beneficiary preferences and identify key challenges and barriers as well as enablers for PHR use for this population.  Additionally, utilization data analysis revealed beneficiary PHR usage patterns and frequency of use, and suggested preferences for PHR functions within this population. 

For each evaluation activity, NORC submitted an Institutional Review Board (IRB) package internally to assure the evaluation plan and methods developed met NORC’s requirements for appropriate and ethical research standards.  Once official approval to conduct each evaluation activity was received from the IRB, NORC began the recruitment process. NORC contracted with Alan Newman Research (ANR) to assist in all recruitment activities and meeting logistics.

All instrumentation for data collection activities were developed in close consultation with ASPE, CMS, Dr. Patricia Flatley Brennan and the members of the PHR Technical Expert Panel. Below, we describe specific processes related to design, data collection and analysis for each evaluation component beginning with beneficiary discussion groups.

Discussion Groups with Beneficiary MyPHRSC Users.  NORC conducted two in-person meetings with beneficiaries considered ‘users’[3] of MyPHRSC.  In close collaboration with CMS, ASPE, and key management staff at QSSI, NORC secured an initial selection pool of beneficiaries who had used the PHR one or more times since registering for the tool and who were physically located within the Columbia, SC region. 

NORC developed a pre-discussion screener to provide context for the discussion groups and to prepare beneficiaries to recall PHR functions and use.  ANR assisted in fielding the brief pre-discussion screener, which asked beneficiaries when they signed up for the PHR; how often they used the PHR; how they found out about the PHR; and any issues or concerns they had in using the PHR. Beneficiaries were also asked to spend time using the PHR prior to the discussions in order to allow them to be prepared to share their experiences with the PHR and to ensure they could accurately recall their experiences.  The pre-discussion screener is included in Appendix 9 while the full discussion group guide used for this activity can be found in Appendix 5. The topics covered in the discussion group guide include:

• Overall Computer Experience
• Perceptions of PHR Utility
• Perceptions of PHR Usability
• Perceived impact of PHR Use on Health Status
• Consumer Satisfaction with PHRs

Two ninety-minute group meetings were held in Columbia, South Carolina on September 11^th and September 12^th, 2008.  NORC staff documented each session, including all the individual comments from beneficiaries.  Each meeting was led by two facilitators accompanied by one note taker to record the conversations.  The first meeting was attended by four beneficiaries, while eight beneficiaries attended the second meeting, yielding a total of 12 participants. 

NORC compiled meeting notes and any discrepancies were resolved by coordinating notes with audio recordings taken during each meeting.  Final notes from each discussion group were reviewed by discussion topic and responses.  NORC performed analysis of the meeting results to identify emergent categories or recurring themes relating to the key research questions, particularly, those related to use, usability and utility of the PHR from the patient perspective.

Semi-Structured Telephone Interviews with MyPHRSC Nonusers. Nonuser[4] perceptions of the PHR were gathered through nine semi-structured telephone interviews held from November 20^th to December 12^th, 2008.  QSSI provided NORC a list of beneficiaries who had not logged in since registering for MyPHRSC and who were physically located in the Columbia, South Carolina region.  QSSI provided NORC with a list of names and contact information for 50 beneficiaries considered nonusers.  NORC placed telephone calls to 12 beneficiaries on this list to assess their interest in participating in these discussions, and 9 of 12 beneficiaries signed up to participate.

The approved semi-structured interview guide for the nonuser interviews covered the major topic areas listed below.  The full semi-structured interview guide is included as Appendix 6.

• Overall Computer Experience
• Perceptions of  Potential PHR Utility
• Perceptions of PHR Usability
• Perceived impact of PHR Use on Health Status
• Consumer Satisfaction with PHRs
• Reasons for not using the PHR

NORC staff documented each beneficiary telephone interview, carefully recording all responses.  Each interview was led by one facilitator accompanied by one note taker to record the conversations.  NORC performed analysis of the interview results to identify emergent categories or recurring themes relating to the key research questions, particularly, those related to usability and usefulness of PHRs from the beneficiary perspective.

Observations of Beneficiary PHR Users. NORC conducted the user observations with MyPHRSC pilot participants in order to gain context-based insights into how PHRs fit into the beneficiaries’ home lives, their relationships with health providers, as well as into the overall healthcare delivery process. This task leveraged a set of social science techniques commonly used in the private sector to create a unique data set that added a strong complement to the other evaluation activities.

The goals of the user observations were as follows:

• Examine the key usability components of MyPHRSC
• Examine the key utility components of MyPHRSC
• Uncover the key issues, concerns and perceptions related to MyPHRSC use and adoption
• Explore the impact of the PHR on patient-provider interactions, patient self-care and self-management

Beneficiary contact information was provided by QSSI.  The method for participant selection for this task is not designed to obtain a representative sample.  However, this is not critical to the approach taken in this task, or as important as obtaining productive observation sessions with the people who were selected. NORC recruited beneficiaries from three groups: 1) Beneficiaries who were eligible for Medicare due to age or disability; 2) Beneficiaries who had primary care responsibility for themselves or had a caregiver; and 3) Beneficiaries who actively participated in the discussion groups or beneficiaries who did not participate. A total of five respondents were interviewed in five sessions over the course of two days. All participants were from the Columbia, South Carolina area.

Participant	Eligibility	Independent or Caregiver	Participant in Beneficiary Discussion Group
Participant #1	Age eligible	Independent	Yes
Participant #2	Age eligible	Independent	Yes
Participants #3	Age eligible	Caregiver	Yes
Participant #4	Disability status	Independent	Yes
Participant #5	Age eligible	Independent	Yes

Beneficiaries were asked to complete a Personal Health Record Booklet prior to the user observation. This booklet included an informed consent form and the following three activities: 1) Use your personal health record; 2) Explain where you keep your health information; and 3) Write a Journal.  

The final discussion guide covered the major topic areas related to beneficiary background, comfort with technology, health information management, current use of MyPHRSC and key challenges. The full discussion guide is included as Appendix 7. 

During the observation, participants were asked to demonstrate how they would use the PHR in various situations. For each participant, NORC also conducted data entry tasks to identify any variability. Although not every participant viewed every page of the PHR, all were asked to view the health record summary and to conduct a search on medical information. Most participants also entered medication information into the PHR while some were asked to complete a Wallet Card.

Given that MyPHRSC was a novel piece of technology, it was thought that beneficiaries might have difficulty relating to a direct question about their general impressions on using the application. NORC therefore used a social marketing technique of emotional affinity to provide a framework for participants to share some of the emotions they experienced when using MyPHRSC. Participants were asked to select from a set of images, which acted as a surrogate for the array of emotions they might experience when using the PHR. Although not part of the initial goals for the user observations, understanding what emotional reaction MyPHRSC provoked in users was a supplemental question that emerged over the course of the study.

Three staff from NORC conducted the user observations, including two co-facilitators who led the protocol and one note-taker who took detailed notes. All sessions were audio recorded.

Discussion Group with Providers.  Provider perceptions of PHRs were gathered through one 90-minute discussion group held via WebEx and teleconference on November 13^th, 2008 with a total of 9 providers.  NORC contracted with ANR to assist in recruitment of providers practicing in South Carolina with a practice population base of 25% or more Fee-for-Service Medicare beneficiaries.  ANR also assisted in fielding a brief pre-discussion screener.  The sample screener is included as Appendix 10.

The approved provider discussion guide addressed the major topic areas as listed below.  The full discussion guide is included as Appendix 8.

• Experience using Health Information Technology in practice
• Overall Perceptions of PHRs
• Perceptions of PHR Utility

NORC staff documented the session, including all individual comments from providers.  The meeting was led by two facilitators accompanied by one note taker to record the conversations.  In total, nine providers joined the meeting by teleconference and WebEx. 

NORC performed analysis of the meeting results to identify emergent categories or recurring themes relating to the key research questions, particularly those related to provider perceptions of use, usability and usefulness of PHRs.

Analysis of Usage Data.  NORC analyzed one year of registration and usage data from the PHR application to describe characteristics of beneficiaries that registered for and used MyPHRSC, as well as to understand broad patterns of use for MyPHRSC. The quantitative analysis was intended to be used in conjunction with the qualitative analysis, as the findings provided further avenues of research to be explored with more comprehensive data.  The analysis provides information relevant to the following key research questions:

• Key utility and usability components of MyPHRSC.  How frequently do users return to the PHR? What are the general patterns of use? What are the key features of the PHR? Which PHR functions are used most and least often by beneficiaries? Based on the usage data, we can also make inferences in terms of PHR usability.
• Impact of the MyPHRSC on disease management and patient self-care. What is the prevalence of chronic conditions among the population? Are there any differences in use between registrants with chronic conditions and other users?

To guide the analysis of the usage data, NORC developed a set of three hypotheses. The development of the initial set of hypotheses considered the overall research objectives of this evaluation, key findings from the PHR environmental scan and a preliminary analysis of the usage data.  The hypotheses were also reviewed with ASPE, CMS and the PHR Expert Panel for feedback. The three hypotheses were:

Beneficiaries with chronic conditions or diagnoses are more likely to use MyPHRSC than beneficiaries without such conditions. Previous research has shown that these beneficiaries exhibit a greater need for and higher interest in PHRs.

 Table 2.     Data Table Summary over Different Periods, including Daily, Monthly, and Quarterly Example

Reporting Period Variable Name Daily User ID User log in count
Registration Monthly User ID User log in count Quarterly User ID Total views for each MyPHRSC page Not time-dependent User ID Gender
Date of birth
Illness/condition diagnosis

Variables of interest. The analyses involved predictor variables, such as age and gender, and dependent variables, like user log-ins and page views.  Some dependent variables were constructed from the data provided.

PREDICTOR VARIABLES

Gender, age, and illness/condition diagnoses were selected as predictor variables since they were independent of MyPHRSC.  Age as of the midpoint of the pilot (September 30, 2008) was calculated from the beneficiary’s birth date.  Information on the illnesses/condition diagnosis for each user was based on a translation of International Statistical Classification of Diseases and Related Health Problems (ICD-9) diagnostic codes to Systematized Nomenclature of Medicine—Clinical Terms (SNOMED CT) descriptions that the participants might recognize.[6]  The illnesses/conditions diagnosis list contained only those conditions for which Medicare processed a claim containing that particular code.  MyPHRSC automatically included the Medicare claims descriptive terms, and users could enter additional conditions as needed.

Some MyPHRSC users were not the actual Medicare beneficiaries.  Authorized representatives were those who a beneficiary designated to be able to view their health information on MyPHRSC. These individuals were assigned a unique ID separate from the beneficiary’s user ID. Since NORC wanted to assess the use of the PHR by Medicare beneficiaries, authorized representatives were excluded from analysis.

DEPENDENT VARIABLES

The two dependent variables included:

1. Login category:  never, single, or multiple
2. Months used

The two dependant variables were less straightforward to develop.  First, it was necessary to operationally define “use” of MyPHRSC.  Based on the data available “use” could be defined as any of the following:

• registering for MyPHRSC
• user log ins to MyPHRSC
• logging in once
• logging in more than once
• viewing particular pages in MyPHRSC

All beneficiaries who participated in the pilot had to register for MyPHRSC in order to participate.  For the purposes of this analysis, dependent measures were constructed primarily from the log-in data.  For the first dependent variable, registrants were initially classified into three categories that differentiate between non-use, one-time use, and repeat-use of MyPHRSC:

• Never logged in to MyPHRSC during the study period
• Logged in to MyPHRSC one time during the study period
• Logged in to MyPHRSC more than one time during the study period

Additionally, a dependent measure was constructed that tabulated the total number of months during the study period in which a registrant logged in to MyPHRSC at least once in that month.  Rather than aggregating total logins, which may have been clustered in a single or only a few months, this measure allowed for an investigation of repeated use of MyPHRSC over time, and may have more accurately represented sustained usage.  We can imagine, for example, that a beneficiary might have logged in several times in the first month, lost interest, and never logged in again.  This is different than a beneficiary who logged in once or twice a month every month to keep track of prescription drug use or to review processed claims.

Finally, the number of times a beneficiary viewed each page of MyPHRSC was included as a variable in this analysis.

DATA ANALYSIS

The data analysis strategy revolved around developing descriptive information about different kinds of registrants, as well as testing the proposed hypotheses.  For all variables, whether predictor or dependent, we provided a frequency, or count, of the number of occurrences of that variable.  These frequencies provided us with a broad-brush understanding of the beneficiaries who utilized MyPHRSC and general findings about their usage patterns.

Because age and gender both have implications for technology use, we conducted bivariate analyses of categorical variables separated by age and gender and tested these associations for significance.  A p-value of less than 0.05 indicates that, if the null hypothesis is true, there is less than a 5% chance of obtaining the observed result or one equally extreme. The bivariate analyses provided associations between variables — in the example above, thinking about age ranges for men and women and their usage might be of more interest than simply thinking about gender or age and usage.  To understand the combined effects of the variables of interest, we conducted multivariate analyses. 

To test the hypothesis that beneficiaries with chronic conditions or diagnoses were more likely to use MyPHRSC than beneficiaries without conditions, we conducted multivariate logistic regression predicting MyPHRSC use by illness/condition diagnosis, while adjusting for age and gender.  This adjustment for age and gender was made since the elderly are more likely to have chronic diseases and have different usage patterns, and women are likely to have different usage patterns than men.

To test the second hypothesis that women are more likely than men to use MyPHRSC, NORC utilized a multivariate logistic regression.  Use of MyPHRSC was the dependent variable, and gender was the main explanatory variable. NORC adjusted for age.  A similar strategy was used for the third hypothesis — that younger beneficiaries are more likely to use MyPHRSC than older beneficiaries.

To test the third hypothesis that younger beneficiaries are more likely to use MyPHRSC than older beneficiaries, NORC used multivariate logistic regression, and adjusted for gender since women may have different usage patterns than men.

Phase 3: Final Report.

The final phase of the evaluation consisted of a cross cutting synthesis of all the evaluation activities and developing the final report.  All information collection activities and analysis conducted in phase 1 and phase 2 of the research informed conclusions developed in the final report. The final report summarizes all evaluation findings, answers key research questions, and makes suggestions for future research activities.

Study Limitations.

The findings presented in this report have several limitations in terms of the generalizability of study findings, limitations of the MyPHRSC tool and usage data constraints. These limitations are discussed in further detail below.

Applicability of Results. The population for this study consisted of fee-for-service beneficiaries in South Carolina that self selected to use MyPHRSC. Therefore, this population is not representative of Medicare beneficiaries as a whole and the findings outlined in this report may not be applicable to the broader Medicare community.

This pilot involved only a single PHR product. Consequently, the study evaluates the experiences of beneficiaries using MyPHRSC.  There are many different types of PHRs offered by health plans, vendors and providers, and the results of this evaluation may not inform the experience of beneficiaries who are using PHRs that are tethered to Electronic Health Records (EHRs) and that are not claims-based or offer more information than claims. Additionally since the data in the PHR was generated entirely from claims there were issues of accuracy and/or reliability of the information.  Respondents to a study conducted by Grossman et al. noted that claims data had inherent weaknesses relative to clinical data[28].

Small Sample Size.  NORC experienced numerous obstacles in recruiting participants for the various evaluation activities and worked with the CMS contractor, QSSI, in all recruitment efforts. Due to challenges in recruiting participants, the sample sizes for the qualitative activities of the evaluation are limited.

Limitations of MyPHRSC.  The evaluation also revealed several limitations of the MyPHRSC tool itself. Firstly, since our population was fee-for-service beneficiaries, the PHR did not offer a comprehensive source of information to facilitate coordination of care across providers[7]. Specifically, this pilot did not involve providers and the PHR did not provide access or tools specifically for providers.  MyPHRSC did not include features to allow beneficiaries to communicate electronically with their providers, such as secure messaging or emailing. Such features may serve to address communication barriers between physicians and patients and contribute to physicians’ ability to provide a more personalized level of care. PHRs with the capability to directly communicate with providers can also serve to improve the speed and quality of communication between physicians and their patients. Additionally, MyPHRSC did include a mechanism for providers to integrate the PHR with an EHR system. Features such as these provide several advantages and can have the further effect of increasing the value of the PHR.[29] [30]

An issue beneficiaries identified during discussion groups and the observational studies was that the PHR lacked comprehensive, longitudinal health information.  MyPHRSC was pre-populated with two years of claims data, however many beneficiaries desired an extended or lifetime view of their health care histories. Additionally, MyPHRSC excluded information on laboratory results, and beneficiaries indicated they desired a PHR to view their test results and track results over time. Beneficiaries participating in the pilot study indicated that auto-population of laboratory results would be a useful supplement to claims data in MyPHRSC.

During the pilot, a federal rule prohibiting the transmission of Medicare Part D data prevented access to medication data for pre-populating MyPHRSC. As a result, medications were not listed in MyPHRSC, thus requiring users to manually enter that data into the PHR. Beneficiaries participating in the pilot indicated that auto-population of medications would have been very useful and would have made the medication function less challenging to use.

Palmetto facilitated importation of CMS claims data into the PHR and identified limitations in pre-populating claims data from their existing claims system into MyPHRSC. Utility and usability could be improved in designing a PHR to incorporate a variety of data types including claims and clinical data, such as seen in electronic medical record systems.  

Provider Discussion Group.  Given that NORC only spoke with a small group of providers in South Carolina, findings from this discussion may not be applicable to all providers.  Participants in this discussion may not have represented the opinions of the whole group, or of future groups of providers.

Usage Data Constraints. The registration and usage data analysis is constrained by a number of limitations.  First, the South Carolina Medicare beneficiaries who registered for MyPHRSC self-selected into the pilot by choosing to register. This results in a biased study sample.  Beneficiaries who chose to register may have been more technologically inclined than the general beneficiary population, or may differ in other respects.  As a result, findings cannot necessarily be generalized to the entire South Carolina or broader Medicare population. 

Second, the data on individual users’ page views was too coarse for meaningful analysis.  Because page view data was available only as an aggregated quarterly total for each user, it was not possible to tie pages viewed to specific logins.  Furthermore, because so few page views exist for many users in our data collected, there is insufficient statistical power to perform comparisons in many cases.

Third, session data on individual users' navigational path (i.e., the sequence of pages viewed) when using the PHR was not available.

Fourth, information on illnesses/conditions was available only as lay user descriptions which may have resulted from a number of different underlying diagnostic codes.  It is not possible to determine with certainty that every beneficiary identified in the data as having a particular condition does have that condition in the same way it is defined elsewhere and in the literature.  Similarly, it is not possible to determine with certainty that users who are not identified in the data as having a particular condition do not have that condition.

Although limited in the generalizability of the results, this study expands the limited knowledge base regarding PHR use and user-friendliness, expands the limited understanding of how the elderly relate to technology, and will aid CMS in determining their role regarding PHRs and beneficiaries for future projects.

Results

Below, we present detailed findings from each evaluation activity. We begin by sharing results from background discussions with MyPHRSC contractors. Next, we provide a detailed analysis of each of the qualitative and evaluation activities.  This is followed by an overview of pertinent results from the usage data analysis.

Findings are organized as follows:

• Discussions with MyPHRSC Contractors
• Discussion Groups with Beneficiary MyPHRSC Users
• Semi-Structured Interviews with Beneficiary MyPHRSC Nonusers
• Observations of Beneficiary MyPHRSC Users
• Discussion Group with Providers
• Usage Data Analysis

Discussions with MyPHRSC Contractors

NORC conducted telephone discussions with three key informants from companies involved in implementing the pilot program. These discussions, including experiences both before and during this project with respect to development and functionalities of the MyPHRSC product, the data lifecycle, standards, security, outreach and challenges experienced, served to inform NORC’s understanding of MyPHRSC and provided context for conducting further evaluation activities.

Data lifecycle

The discussion with Palmetto indicated that only fully processed claims data were populating the PHR.  Given that claims had to be submitted by providers and then complete claims processing, there was a certain level of unpredictability regarding when information would be available in MyPHRSC.  Patients may have encountered a lag between a healthcare encounter and when the information became available in the PHR.

While HealthTrio had prior experience with importing claims data into PHRs, discussions with Palmetto and HealthTrio revealed that they met several times to identify and coordinate the claims fields that would be used to populate the PHR with the appropriate data.  Based on these discussions, Palmetto developed and implemented a specific extract program to create the necessary format required to populate data into HealthTrio’s existing PHR product.

Once a beneficiary signed up for MyPHRSC, HealthTrio notified Palmetto. Within 48 hours of a beneficiary signing up, Palmetto made available 24 months of Medicare claims data for each beneficiary and populated each MyPHRSC account.

Palmetto and HealthTrio reported that no claims data was altered. Therefore, any inconsistency or inaccuracy of the information displayed in the PHR was a function of incorrect/inaccurate information being submitted on the original claim. Additionally, HealthTrio indicated that, in order to make medical terms more understandable to users, they convert International Classification of Diseases, 9th Revision (ICD-9) codes and Current Procedural Terminology (CPT) codes to Systematized Nomenclature of Medicine (SNOMED^[8]) terms.

Security

All the informants reported that CMS has stringent data security requirements by which the contractor and both subcontractors abided.  To ensure that HealthTrio met the CMS security requirements, they conducted a security audit and consequently addressed issues related to documentation of security procedures. CMS initially requested three-zone architecture to meet Federal Information Security Management Act (FISMA) security guidelines. HealthTrio worked closely with CMS to ensure the appropriate level of security standards and safeguards were in place.[31]

Outreach.

In addition to being the prime contractor, QSSI was also responsible for carrying out all outreach to the Medicare beneficiary population to encourage registration and use of MyPHRSC. QSSI conducted statewide advertisement and outreach that targeted three types of events: workshops, health fairs and presentations. QSSI recommended developing solid partnerships with local organizations to help with marketing, citing its relationship with South Carolina’s Lieutenant Governor’s Office on Aging as instrumental to their ongoing efforts to reach the Medicare population of South Carolina. QSSI indicated that the provider outreach was more challenging since providers must be given PHR access through their patients and consequently seemed to have less of a vested interest in this pilot.

Discussion Groups with Beneficiary Users of MyPHRSC

When screening for the user meetings, the selected participants indicated a broad range of experience levels in using computers and in using the PHR.  Beneficiaries reported first hearing about the PHR between 1 month and 6 months prior to September, and first signed up for the PHR between 1 week and 4 months prior to September.  Expectations of MyPHRSC varied. Many beneficiaries reported that they did not know what to expect of the PHR and wanted to simply learn more about the tool, while some reported signing up in hopes that the PHR could help them to keep track of their records (e.g. billing, claims and personal health information). Appendix 9 lists the results from the pre-discussion screener.

In discussion groups, NORC investigated key components of utility (the quality or condition of being useful) and usability (how well users can learn and use a product to achieve their goals and how satisfied they are with that process).  Participants identified computer and Internet experience; key health care priorities; reasons for using the PHR; most valued features and functions; challenges and enablers to using the PHR; key benefits experienced or anticipated; and potential impacts of using the PHR.  Below, we provide a summary of key results and conclusions.

Beneficiaries’ Backgrounds.

Overall Computer Experience.  Beneficiaries were asked if they used a computer on a regular basis; how often they used the Internet; and the kinds of things they used the Internet for, such as online shopping and banking, and searching for health information.  Overall, most beneficiaries in both user groups had high speed Internet connections in their homes; many used the Internet regularly, and most were comfortable using computers.  Many regularly performed activities such as banking and shopping online, while a few cited concerns around security of their information, which resulted in more limited Internet usage.  Most participants also reported that they used the Internet to search for health information. 

A handful of beneficiaries did not consider themselves to be computer-savvy. Many reported maintaining paper medical records, and one beneficiary preferred this method to using a computer. One beneficiary noted the ability to use the PHR, but the spouse would not use it because of difficulty accessing their home computer.

Alternatively, one beneficiary found that the Internet allowed many tasks to be independently accomplished, which otherwise could not have been done without assistance due to a physical disability.  The individual reported using the Internet for blogging, chatting with friends made online, banking, helping family members to organize their finances, searching for health information, ordering prescriptions, and shopping.

Overall, the group perceived the Internet as a useful source of information and beneficiaries reported that they were able to perform a variety of online functions, including online shopping, banking, and searching for information.  Additionally, a handful of participants agreed that using a computer helped them to stay organized. 

Experience Using the PHR.  Overall, beneficiaries reported having used the PHR within one month prior to the meeting; and a handful reported having used the PHR two weeks prior to the meeting.  Participants were divided on the amount of experience they had using the PHR, with some using the PHR on a regular basis and some who had only used the PHR a few times since first signing up.  All participants were eager to learn more about the tool.

Health Care Priorities.  In order to understand why particular functions or features might be important for beneficiaries, we asked beneficiaries what practices they currently had in place to manage their health and healthcare, and what they considered to be their healthcare priorities.  The most commonly cited healthcare priorities included continuity of care, concerns about reducing fragmented care, and self-managing chronic conditions such as diabetes.

Usability of MyPHRSC.

General Perceptions of User-Friendliness.  Most beneficiaries agreed that the PHR was easy to use once they logged into the system.  Overall, beneficiaries would have liked the text in the log-in screen to have a larger font and for the log-in text boxes to be located more centrally on the page.  Additionally, it was noted that the ability to change the font size within the PHR helped make the PHR easier for beneficiaries to read.

Accuracy of Data.  Many beneficiaries noted that the PHR contained inaccurate information, including illnesses listed that patients did not believe they had, and incorrect or unrecognized physicians’ names.  A few beneficiaries indicated that their illnesses were listed as “active” even if they occurred in the “past”, and beneficiaries were generally confused as to how an active illness was defined in the PHR.

“I don’t want my doctor to look at [the PHR] today because it is inaccurate or incomplete, [although] it may improve over time.”

Beneficiary

Sharing their information with doctors was cited as an important utility of the PHR; however, several beneficiaries indicated the utility would be dependent on the accuracy of the information.  Overall, beneficiaries noted that some information was inaccurate, but they would be interested in using the PHR if this information were corrected.

Logging in to the PHR.  Beneficiaries reported a number of difficulties using the PHR; but overall they viewed the PHR as useful. The most significant issues for beneficiaries were focused around signing in to the main site, e.g., user ID or password lockout. 

Many beneficiaries described having trouble signing in the first time, while a few reported it as easy.  In order to log in the first time, beneficiaries often required assistance from the QSSI[9] technical support. Some participants required technical assistance on multiple occasions in order to log in successfully to the PHR.  It was challenging to enter the information for registering in the required proper format, and a few noted the system was quite sensitive.  For example, MyPHRSC does not allow beneficiaries to choose their own ID, and proved challenging.

Much of the frustration around logging in for the first time revolved around using and remembering log-in user IDs and passwords.  MyPHRSC provided a temporary password for beneficiaries, but beneficiaries often had trouble entering this correctly.  One beneficiary noted that the password was in lower case but the user ID was in all capital letters with no spaces, requiring significant effort to complete correctly each time.  This proved particularly frustrating since beneficiaries who used incorrect information a certain number of times were locked out of the system for 24 hours.  Using and remembering their user IDs and passwords created a significant barrier to logging in to the system.

Changing Passwords after 90 Days.  Some beneficiaries noted that they worked with technical support to log in properly the first time, but were unaware they had to change their password every 90 days. This confused them. Although security of health information is important to beneficiaries, when asked if it was worth having to change passwords every 90 days to ensure security, beneficiaries were divided, with some citing memory troubles. Beneficiaries all agreed that changing passwords every 90 days was challenging.

Technical Support Experience.  NORC probed as to why participants continued to persevere despite their troubles of getting into the system.  Many of the beneficiaries found that the technical support provided through e-mail and a toll-free number was helpful.  One beneficiary noted: “Instead of calling, I e-mailed the contact, and the helpdesk was tremendously responsive and solved my problem quick.”  Others agreed, suggesting, “The courtesy from [the helpdesk] is tremendous.”  One beneficiary persevered because of the ability to view the spouse’s data first, stating, “If I hadn’t seen my [spouse’s] data I might not have pursued.  I was frustrated but once I called, I spoke to [the helpdesk and] they worked hard to make sure my data got into the system.”

In spite of the usefulness of the toll-free (800) number and e-mail support, many reported receiving poor support when they called the second toll-free (877) number.[10] Some indicated the technical help through the latter number included inconsistent information, and some were further directed to call Medicare’s general toll-free number (a third number), proving most challenging.

Delay in Population of Claims Data.  A key challenge for beneficiaries included the time delay from an encounter with a provider until seeing the information in MyPHRSC.  It is important to note that beneficiaries generally were not aware of the delays in submitting and processing claims, and these were issues outside of the PHR pilot.  Some beneficiaries experienced a paper summary of CMS claims information arriving in the mail relatively quickly after a visit or other medical event. It was felt that, in order for the information in the PHR to be useful for them, it would need to be timelier. 

Confusion Using Functions.  Some beneficiaries reported that different MyPHRSC features were confusing to use, and this made using the PHR challenging. Beneficiaries mentioned that saving self-entered information for medications and allergies did not update and persist to the next use as was expected. Another noted that the PHR did not allow more than one item to be entered into the calendar, while others preferred a paper calendar with all pertinent events (beyond health care). 

Utility of MyPHRSC.

Most Valued Features and Functions.  Many beneficiaries commented that the Wallet Card was particularly useful for them and they especially liked the ability to print this to carry with them.  One beneficiary commented: “I thought the card would be good to have when we travel.” Most agreed that having this information on hand while traveling would be useful, and one couple mentioned they would like to have it with them on their next vacation.  Many beneficiaries also noted that Medicare had provided them with an index card but it lacked the depth of information in the PHR wallet card. Participants thought it was particularly useful to carry a concise list such as medications, surgeries and past hospitalizations, so as not to take up too much space. 

Beneficiaries were also impressed with the completeness of the information in the Health Record Summary and Claims pages. They thought the PHR provided more comprehensive information than paper summaries from other health plans like TRICARE and Medicare, which include only the cost and/or dates for services and the physician’s name.  Participants also noted that paper summaries can be hard to read, and liked that they could easily adjust text size.  One beneficiary noted, “When you open [the PHR] you see the diagnosis, the doctor name and the date and it is clear to understand.”

Participants reported that it was useful simply to view all their medical claims from the past two years in one place.  Although many beneficiaries also maintained paper records, they noted that having access to 2 years of claims summaries emphasized the number of medical encounters they experienced.  This was particularly beneficial for those with short-term memory deficit.  One beneficiary noted that the calendar feature could be helpful for tracking appointments and medications in one place.

Reasons for Using.  Many beneficiaries initially logged into the PHR out of curiosity. Some participants used the PHR to update and print their wallet cards, and a few reported using the PHR to print the healthcare summary. One couple used the PHR to enter prescriptions that Medicare would not pay for so it could be conveniently tracked.  Many participants used the PHR to view their past claims information.

“I have several doctors but it is up to me to maintain my health records.  I cannot get [my doctors] together at one time so I need to keep them informed.”

Beneficiary

“It is always nice to have a list of all the medications you are taking because [doctors] always ask that question.  If you bring all [of your medication bottles] in to the doctor, you have to carry a sack full of these on your shoulder.”

Beneficiary

Key MyPHRSC Benefits.  Although participants reported varying levels of experience using the PHR, a majority agreed the PHR could be a useful tool for them to manage their health or healthcare.  Beneficiaries thought that the PHR helped them keep track of historical medical events (e.g. longitudinal record), and many reported that having the information in the wallet card at hand while traveling would be very helpful for them.  The ability to have all of one’s health information stored in one place was also seen as a potential benefit of continued use of the PHR.  Although participants had not shared their records with providers, they thought the PHR could be more useful if also used by their providers.

One participant found that having the medical history gave a new view into specific health conditions. Others agreed that they liked having a history of their records with one beneficiary suggesting that sharing this information with providers could be valuable: “I like that I can add my history, but I really like that, if I have to get a new doctor, I can look at the form that shows everything that happened to me and give this to the new doctor. This is good, if you have to move, as a history.”

Management of Personal Health Information. Many beneficiaries reported that they believe they are the primary owners of their own health information, and they are responsible for managing their own care, so they should “own” the personal information stored in the PHRs.  One participant commented, “I don’t think doctors will pay attention to what is in there, it is key for me and my [spouse] and my family to know what is wrong with me and if I get to a place where I cannot convey problems they have a history to look at.”

Although there was some disagreement as to who should own the PHR, beneficiaries generally agreed that it is important to remain informed about one’s own health and health care through appropriate communication.

Privacy and Security.  Many beneficiaries indicated that they were concerned about the privacy and security of their health information in the PHR, but would be willing to share the PHR with trusted sources such as a provider or caregiver because this would be valuable for them.  A few participants indicated little concern about privacy and security. Providers can gain value from the PHR information shared, which “is more important than [being concerned] that you are putting it out there and someone may see it.”  Beneficiaries reported that they would not want their personal information used against them, and many did not realize the potential impact of fraudulent claims. 

Potential Impact of the PHR on Health Status.

Many participants indicated they had not interacted in the PHR enough to suggest any impact on their health status.  However, one beneficiary noted finding a diagnosis in her PHR that the doctor had not discussed. The beneficiary indicated that having access to the information prompted a follow-up with a gastroenterologist.

Some beneficiaries reported hopes that the PHR could help handle some of the difficulties around a fragmented care system. Provision of care from numerous providers often requires beneficiaries to take a more active role in scheduling their different appointments, checking interactions of drugs prescribed by different doctors, and remembering historical medical events, allergies, and medications to share with doctors who may not have this information.  Many thought that the PHR could help coordinate information and, consequently their care, with beneficiaries believing the use of the wallet card had a most positive impact.  Additionally, several reported that they planned to speak with their providers about the PHR in the future.

Consumer Satisfaction with the PHR.

Overall Impressions.  Generally, beneficiaries thought the PHR could be useful for them, but with modifications as noted below.  Many reported viewing the PHR quite positively once they logged in, even in cases where they experienced technical difficulties accessing the PHR. One beneficiary stated, “I was impressed. When you open the PHR, you see the diagnosis, the doctor name and the date, and it is clear to understand.”

Suggestions for PHR Developers.

Self-Entry of Information.  Generally, participants reported that they did not want to manually enter information into the PHR, as this was particularly time consuming for beneficiaries. 

“The crux is this system is only helpful if it has a lot of information from people other than us.  [Currently] it is depending on us entering information into MyPHRSC and few people will take the time to enter information and gather information from doctors to put in. The goal should be to aim at getting it from the Medicare system or health providers.”

Beneficiary

Medications and Decision Support.  Many beneficiaries suggested the PHR would be more useful to them if their medication data was entered for them.  One beneficiary explained: “I would like a complete list of medications, immunizations, and conditions that a doctor would have access to.”  Most beneficiaries also reported drug-drug and drug-allergy interactions checking would be very helpful for them.

Integration with Doctors.  The majority of beneficiaries believed the PHR would be the most valuable if doctors used it.  One participant noted: “If I have blood work with one doctor and a couple of weeks later I have an appointment with another doctor, it might be for the same test. One advantage to this is [provider] could look at the test and see the test had already been performed.”  Another beneficiary explained willingness to self-enter more information if doctors were to use the PHR.

The group discussed the potential benefit of having the PHR interact with an EHR and some thought this would be useful.  One participant commented, “It depends on interfacing with all of the providers. Having three of your providers enter information in the PHR and your 4^th provider not use it, it would not work.”

Desired Amount of Information.  The majority of beneficiaries agreed that they would want as much information as possible at hand, with some noting that the full medical record would be of particular use to them. 

Conversely, when asked if they would want diagnostic results like an X-ray or MRI to appear in the PHR, the group was quite divided.  Some reported that they wanted all of their information available, while most agreed that this would be too overwhelming and they would prefer only targeted information be accessible.  Many felt comfortable with their doctors maintaining medical records with test results as an alternative.

Time span of Claims Information. Many participants suggested that 2 years of claims data was not a sufficient record for them and that they would like the history to go back further in the lifetime. A few suggested a lifespan history would be useful. 

Trending Laboratory Results.   Overall, participants would like to be able to see the results of their tests.  Many also noted they would want to track trends in their laboratory test results.  One participant explained, “If it showed you a string so you could see trends in things like cholesterol, blood sugar, and trends even your doctor doesn’t see, that would be useful.” 

Pay to Use the PHR.   Most beneficiaries reported they would pay for a PHR if it contained the functionalities and components most desired.  One participant commented: “If it was a valuable tool for me and my doctor to access, I would pay for it.” One beneficiary who was particularly enthusiastic about the PHR was unable to afford to pay for MyPHRSC but wanted to continue using it.

Semi-Structured Interviews with Beneficiary MyPHRSC Nonusers

In discussion groups, beneficiary nonusers of MyPHRSC identified computer and Internet experience; key health care priorities; reasons for registering for MyPHRSC; reasons for not using the PHR; suggestions for PHR developers; and comments regarding the usability and utility of MyPHRSC. Below, we provide a summary of key results and conclusions.

Beneficiaries’ Backgrounds.

Overall Computer Experience.  Many nonusers reported they had high speed Internet connections in their homes, used the Internet on a daily basis, and indicated their relative comfort with computer use.  Many regularly performed activities such as banking, investment tracking, checking the news and shopping online; however, two individuals did cite concerns in regards to the security of their information. In both of these cases, beneficiaries were hesitant to share their personal information on the Internet.  Most participants also reported that they had used the Internet to search for health information. 

Two beneficiaries used the Internet approximately every two to three weeks; both individuals indicated that their Internet usage was more frequent due to an adverse health event.  Many beneficiaries reported maintaining paper medical records. Due to concerns about the security of electronic information, one of the beneficiaries preferred maintaining paper medical records. In one case a beneficiary indicated using the PHR for a significant other since this person was not able to use a computer.

Health Care Priorities.  In order to understand why particular functions or features might be important for beneficiaries, we asked beneficiaries what types of actions they took to manage their health and healthcare, and what they considered to be their healthcare priorities. The most commonly cited healthcare priorities included medication management, reduction of healthcare costs, tracking claims and benefit information, preventative health information, continuity of care, concerns around fragmented care, and self-managing chronic conditions such as diabetes.

Reasons for Signing Up.  Beneficiaries reported different reasons for signing up. These included curiosity about a tool like a personal health record, and the desire to access all their health information in a single place. Many participants had hoped to use the PHR to view their past claims information and track their medication and health history. Most indicated signing up for the PHR in hopes that it would assist them in tracking their medical history, and in turn their health.

Reasons for Not Using PHR. There were several reasons that participants did not use the PHR after signing up. The main reason involved forgetfulness; however, other reasons included lack of provider interest in the PHR and concerns over security of their information online. The  most prevalent reasons for not using the PHR included forgetting about the availability of the tool, being too busy with other things, forgetting the password or the password expiring, and not considering the use of MyPHRSC to be a part of their normal Internet workflow. All but one of the participants indicated interest in using and learning more about MyPHRSC.

Logging in to the PHR. The most significant challenges for nonuser beneficiaries were focused around signing in to the site.  Beneficiaries reported a number of difficulties in logging into the PHR, however, most indicated that they would attempt to log in again after the interview.  Many beneficiaries expressed having trouble signing in the first time, while a few reported logging into the PHR was easy.  Nonusers were asked to log in to the PHR prior to their interviews in order to become familiar with the PHR, and many required assistance from the QSSI technical support to log in.  Much of the frustration around logging in for the first time revolved around using and remembering log-ins and passwords.  MyPHRSC set up a temporary password for beneficiaries, but beneficiaries often had trouble entering this correctly. Remembering their user IDs and passwords created a significant barrier to logging in to the system.

Confusion Using Functions.  Some beneficiaries reported that different functions were confusing to use, and this made using the PHR challenging. Beneficiaries mentioned the saving of self-entered information on medication and allergies did not update the information as expected. Several individuals indicated that the health calendar would be useful; however, some of them indicated that their physicians kept them updated on their appointments, and thus a health calendar would not be useful.

Usability of MyPHRSC.

Although NORC requested beneficiaries log in to the PHR prior to their interviews, many beneficiaries had limited experience using MyPHRSC.  NORC enquired about initial impressions when beneficiaries logged in for the first time.

General perceptions of User-Friendliness.  Most beneficiaries agreed that the PHR was easy to use once they were able to log in.  Overall, beneficiaries would have liked the text in the log-in screen to have a larger font and be more obvious.  Additionally, it was noted that the ability to change the font size within the PHR helped make the PHR easier for beneficiaries to read.

Utility of PHRs.

Most Valued Features and Functions.  Given the fact that the non-user group had limited exposure to MyPHRSC, we asked participants which features and functions they would find most useful in a PHR generally.  Individuals indicated that they would find it useful to track medication history, appointments, and symptoms in an online format. Participants thought it would be useful to print a list of medications, lab results, symptom diaries, surgeries and past physician interactions and hospitalization that they could present to their current providers or to new providers in the event that a health condition would necessitate such a visit. Most participants also indicated that physician access to their PHR information online would be very useful. Additionally, participants indicated that a PHR might provide relevant information in regards to medication side effects, and further information on health conditions relevant to their particular medical history.

Participants indicated that they would prefer to have the health information automatically populated in the PHR. However, most beneficiaries were willing to enter the information themselves if the PHR was user-friendly, and if the PHR provided information of interest for themselves and for their providers.

Participants reported that it would be useful to have a view of all their medical claims, and many also indicated that their three month explanation of benefit (EOB) for CMS claims was not sufficiently detailed.  Although many beneficiaries also maintain paper records, access to claims summaries online was a priority for most beneficiaries. Some beneficiaries who did sign into the PHR indicated that viewing two years of medical claims history in one place made them more aware of the number of medical encounters they experienced. 

Key Benefits Experienced or Anticipated.  The majority of beneficiaries suggested the PHR could be a useful tool for them to manage their health or healthcare.  Nonuser beneficiaries thought that the PHR would help them keep track of their medical history. The ability for all health information to be stored in one place was seen as a potential benefit of continued use of the PHR.  Participants indicated it could be useful if assisting physicians to make more informed diagnoses.

Several beneficiaries indicated that provider access to the information in the PHR, whether through a print-out or through allowing the caregiver online access to their medical history, would be useful.

Privacy and Security.  Only two beneficiaries indicated that they had concerns in regards to the privacy and security of their health information in the PHR, and most indicated that they would be willing to share the PHR with trusted sources such as a provider or caregiver.  The nonuser beneficiaries that did have privacy concerns indicated access and security issues were of the most concern but agreed that further interaction with the PHR might allay these concerns.

Suggestions for PHR Developers.

“If the process isn’t time intensive, and the system is useful to me, I would be OK with entering in the information myself; I would prefer an auto-update system though.”

Beneficiary

Ease of information entry.  NORC asked whether or not entering their own health information would contribute to a loss of some of the PHR utility, and beneficiaries were divided on this issue. Some speculated that they would enter their information into the PHR manually, even if this was a challenging activity for them. 

Medications Information.  Many beneficiaries suggested the PHR would be useful to them if their medication data was entered for them.  One beneficiary explained: “I think you could have, for those who aren’t on Part D, a means to input the medication.”  Most beneficiaries also indicated that the ability to access information on drug interactions would be very helpful.

Integration with Providers.  The overwhelming majority of beneficiaries believed the PHR would be the most valuable if doctors used it.  One participant noted: “I bring my physicians print outs of what I have researched online. It would be really helpful to bring a print out of my medication history, my lab visits, and my physician visits too. I think that way at least my doctor is completely informed.” 

Amount of Information.  The majority of beneficiaries agreed that they would want to have as much information as possible at hand, and some noted that the comprehensive, longitudinal medical record would be of particular use to them.  Some individuals indicated that they wanted to control who accessed the PHR. 

Time Span of Claims Information. Several participants suggested that a lifespan history would be useful. 

Trending of Lab Results.   Overall, participants would like to be able to see the results of their tests over time.

Laboratory Observations of Beneficiary Users of MyPHRSC

Discussions held one-on-one with beneficiaries were focused on understanding beneficiary backgrounds, comfort with technology, health information management practices, their use of MyPHRSC, how this integrated into their normal daily workflow and some of the issues/challenges that users had encountered when interacting with the PHR. Watching beneficiaries navigate through MyPHRSC revealed the successes and challenges they have with using different features.

Beneficiaries found MyPHRSC to contain useful medical information and viewed it as a helpful tool for better managing their health.  Below is a summary of the key observations that emerged over the course of the user observations.

Beneficiaries' Backgrounds.

Beneficiaries provided information regarding their personal background, daily routine, medical status and the involvement of family and friends in their care. Participants were mostly retired and remained active through activities like volunteer work. Many exercised daily, either walking or going to a gym. While the beneficiaries had varying levels of health, all had chronic conditions e.g. hypertension, high cholesterol, diabetes that they were managing.  All beneficiaries received a physical examination annually and many saw specialists regularly as well.

Comfort with Technology.  All of the participants considered themselves to be relatively comfortable using the computer and the Internet. Most reported that the computer was their favorite technology tool. All of the participants in the user observations made use of the computer for staying in touch with friends and family via e-mail, and many played games, shopped, and banked online (although one expressed security concerns about using credit cards online). One beneficiary had a Facebook account, one maintained a Website and online newsletter, and another kept a blog. Most participants lived with their spouse, and there was often one spouse who was the “tech savvy” partner, while the other spouse rarely used the computer or the Internet.

Health Information Management

The health information management practices of the participants were examined. In this context, we specifically looked at general record-keeping practices and how beneficiaries sought medical information. Participants were asked to describe the techniques they employed and the specific challenges and successes they faced in organizing their personal health information. Participants also provided information about their methods for obtaining health information and discussed what impact MyPHRSC had on their health information management practices. The findings outlined below include the information beneficiaries provided over the course of the discussions as well as information from the health journal, which beneficiaries had completed in advance of the meeting.

Participants were categorized in terms of health information management techniques, using the framework described in Moen and Brennan’s article, which included four strategies for organizing health information: just-in-time, just-because, just-in-case, and just-at-hand.[32] These categorizations helped to understand the beneficiaries’ record-keeping techniques. They are defined below:

Just-in-time—when information and/or artifacts are with a household member at most times. Physician contact information is often stored using this strategy.

Just-at-hand—when information and/or artifacts are visible or stored in readily accessible, highly familiar locations in the household. Medication information is often stored in this manner.

Just-in-case—when information and/or artifacts, either personal health files or general health information resources, are kept away but accessible within a reasonable time frame for any future situation. Laboratory test results can fall into this category.

Just-because—when information and/or artifacts about a health concern are brought into and kept in the household but, because of a temporal relevance, no other storage strategy is assigned. Old or out-of-date personal health information is often stored using this strategy.[33]

“I’m not really that organized but they’re just there if we need to get them.”

Beneficiary

Record-keeping.  Much of the personal health information saved by beneficiaries fell into the just-in-case category. Most beneficiaries stored paper records in a file cabinet, bookcase, or drawer within their office or bedroom. Financial and medical information were usually stored near each other in the same drawer or file cabinet. The beneficiaries mostly saved their materials in two locations, which included the just-in-time or just-at-hand categories.  Other information was stored as a collection of older records (commingling medical and tax information) in less accessible locations, such as the attic (the just-in-case or just-because categories). The type of medical information saved varied, although most kept medical bills. Some participants were self-deprecating about their health information management, explaining that they could be more organized. One participant said, “My system is not all that good.”

One beneficiary used both Intuit’s Quicken personal financial software and an electronic personal information manager and relied on both methods extensively for financial recordkeeping. A few participants no longer received paper records from their bank or credit card company and viewed all of their statements online.

Contact information both for health providers and emergency contacts fell within the –just-at-hand storage strategy. This information was often very accessible (e.g., posted on a sheet near the telephone or stored in a telephone book or cell phone). Some participants carried their personal health information (just-in-time), with some participants using the MyPHRSC Wallet Card for that information. Others had homemade documents, either handwritten or typed, with lists of their medications or past medical procedures. While some did not keep information with them at all times, they would update and bring “cheat-sheets” with them when they visited their health providers.

Beneficiaries generally stored their medications in the kitchen and/or bathroom often dividing their morning and nighttime prescriptions between the two locations.

Health Information Sources.  In order to better understand beneficiaries’ health information sources, we inquired as to where they generally obtained information about medical conditions or general health information. Oftentimes, the Internet was a major source of health information for the beneficiaries. Most reported using Google.com to search for medical information, while a few used other popular sites, such as WebMD.com, MayoClinic.com, or their supplemental insurer’s site. Some participants kept general medical reference books in their office or den. Some also received medical newsletters, although they would only keep articles they found useful rather than store the complete issue. Many of the paper copies of these reference materials were stored utilizing the just-because strategy i.e. information brought into the household but of temporal relevance.

Usability and Utility of MyPHRSC Features.

The use of MyPHRSC varied, though most beneficiaries had only logged into the program a handful of times prior to the laboratory session. During the user observation studies, participants were watched as they navigated through different features of the tool and described what they found both useful and challenging about it. Attention was also paid to other specific observations made by the participants regarding the usability (the convenience, effectiveness and ease of use) and the utility (the level of usefulness) of the PHR features.

Below, we outline the successes and challenges of using different features of the tool, as described by the beneficiaries.

Health Record Summary.  Most beneficiaries seemed impressed by all the information while others were overwhelmed by the large list of their illnesses and conditions. One beneficiary commented that the summary was, “much, much more thorough than I could ever imagine,” while another said, “Now you’d look at that and you think I'm pretty sick.” Overall, participants seemed glad that this feature provided a list with greater detail about their health—including dates—than the hardcopy Explanation of Benefits report received from CMS in the mail.

The major dissatisfaction with the Health Record Summary came from beneficiaries not recognizing everything listed on the page. For instance, beneficiaries often did not understand all of the medical terms that were used in the “Problems” list. Some participants also noted that they saw conditions that they were unaware they had been diagnosed with. Others saw health providers’ names they did not recognize or facilities’ addresses that were inaccurate. The beneficiaries suggested that the providers whose names they did not know may have completed lab work or other behind-the-scenes tasks, and that the addresses may be for alternative clinics their providers also visit. One participant saw that both a provider and his spouse were listed twice under the “People” section. This participant noted that the Health Record Summary was not editable, nor was there the ability to edit provider information.

Some information on the page seemed superfluous. For instance, the “Problems” list included a column of the SNOMED and ICD-9 codes, which the participants did not understand. The “Status” column was also unclear and in some cases inaccurate, and often the beneficiary no longer had a certain condition (e.g. cough) that was marked as “Active.” In addition, some beneficiaries were missing information. For instance, one had updated an e-mail address on the User Preferences page, but did not see it on the Health Record Summary page as expected.

Illnesses and Conditions.  Beneficiaries had comments about the Illnesses and Conditions page that were similar to those they had with the “Problems” list from the Health Record Summary. Although the page contained a great deal of useful information, including dates and diagnoses, there were some words that were unclear; for example, a beneficiary reported not understanding what the word “hepatitis” meant. Additionally, some beneficiaries had difficulties using the “add” and “edit” features on this page. When a participant tried to search for a condition to add (e.g., claustrophobia), the error message “No Records Found” appeared. Another beneficiary was disappointed by the lack of detail about a condition within the “Illness/Condition Detail” section, which provides no additional information other than that data from the Illness and Conditions list with participants only able to add “notes”. The participant was hoping to find more information about the listed condition on this page, or possibly see details taken from a provider’s notes regarding the diagnosis.

Medications.  The MyPHRSC pilot program did not have access to Medicare Part D data, so medications were not automatically entered into the PHR. As a result, the Medications page proved to be the most challenging. Some of the beneficiaries were unable to enter their medication information without our guidance. Although not included as part of the original ASPE evaluation, CMS recently downloaded medication information from its Department of Defense partners (TRICARE for Life) in South Carolina. One participant who had both CMS and TRICARE coverage was very pleased to see the medication information listed in the MyPHRSC tool. The TRICARE prescriptions were entered each time medications were filled, which was every three months for this beneficiary.

In order to enter medication information into the PHR, beneficiaries first had to find their medications using the search function on the Medications page.  Searching for a medication involved one of two different paths: 1) identifying the category or subcategory that a medication belonged to; or 2) conducting a top level search. This process was confusing for most of the beneficiaries. They often initially attempted to find their medications using the first path, the “category” search, but were unsure about which category or subcategory their medication belonged. Ultimately, none were able to locate their prescription names using this functionality. Following this attempt, beneficiaries moved to conducting a top level search by typing their medication into the search function at the top of the page. One participant was unsure if entering the generic or the brand name drug was appropriate. The selection of “route,” which offers a drop-down menu of forty-four different routes in the way medication is taken was unclear to some. Many of the terms included in this drop-down menu, such as percutaneous, intraperitoneal, buccal and intrapleural, were confusing.

Once they had located their medication, participants often were unsure how to add it to their medication list. One beneficiary presumed that clicking on the medication would make it automatically appear on the medication list. However, another step was required. After beneficiaries searched and selected their medication from the list, one or more medications appeared in blue under the originally selected drug name. The beneficiaries then had to select the correct medication with the appropriate dosage from that list, which brought them to the “Add Medication” page.

"[The medication page] is not as user-friendly for somebody who is in their late 60s, early 70s."

Beneficiary

The information on the “Add Medication” page was, in the words of one beneficiary, “cumbersome to enter in manually.” First, the page asks beneficiaries to enter the name of the Prescribing Clinician. Attempts to search for clinicians were often unsuccessful. One participant tried to “drag and drop” a provider’s name from the list on the left panel of MyPHRSC. Another searched for the provider by typing in the name using a “Dr. First Last” format, and the provider’s name did not appear. Within the “Provider Search” page, the abbreviation “PCP” was unfamiliar to many beneficiaries. Beneficiaries then entered the information in the dosage, frequency, and duration fields. Most preferred not to complete the “start date,” as it was hard to recollect the actual start date of a medication, particularly in the context of a chronic medication such as an anti-hypertensive.

Immunizations.  Entering immunizations into the PHR proved to be confusing. One beneficiary searched for tetanus, which resulted in a list of ten options from which to select. After selecting one choice, the “Add Immunization” page allows additional information, such as the reason for the immunization and the date of the appointment, to be entered. This beneficiary was confused by the option to enter “activity time” and “end time,” since receiving an immunization occurs at a specific time and was not seen as an event that had a duration.

Wallet Card.  The Wallet Card page was well-liked by beneficiaries, and some cited it as the most useful function of MyPHRSC. Several participants came to the user observation session with their Wallet Cards in their purse or wallet. They found the information on the card useful to take with them to their clinician, and thought that the emergency contact information was especially important to carry around. Despite their overall appreciation of the Wallet Card, participants offered a few suggestions for improving this function.

Firstly, the Wallet Card page did not retain information that the beneficiary had previously entered. Beneficiaries were therefore required to reenter these details each time they wanted to create a Wallet Card. Secondly, beneficiaries were, at times, unclear about the meaning of the terms used in the Wallet Card feature. The first open field of the Wallet Card is “Provider.” One beneficiary did not know what that meant, one was unsure of which doctor to enter, and another thought the PHR should automatically populate the field with the primary care physician. Thirdly, participants took issue with the way the Wallet Card presented information. One participant was unhappy about how the medications appeared on the card, because it did not include the frequency with which the medications were taken. The beneficiary created a work-around by re-entering the entire medication list in the “Other” field under the “Known Allergies” heading, so that the desired details would be included. The beneficiary did not enter it in the actual “Other Information” section (with heading), because this section did not print out within the form. Finally, a couple of participants critiqued the Wallet Card as being too big when printed. It did not fold up to easily fit within a wallet.

Health Calendar.  The Health Calendar page allowed beneficiaries to add activities in preselected categories offered by the MyPHRSC tool. The categories were: health issue, health journal, immunization, procedure, and visit. For the health issue, immunization, and provider categories, beneficiaries picked the applicable event from a provided list of options. For instance, when adding an immunization event, beneficiaries first selected the immunization option from a drop-down list and then would be directed to another page to choose from the various immunizations listed. The PHR also allowed users the option of filtering the calendar to show only events by specific categories.

The Health Calendar was not a popular function of MyPHRSC because beneficiaries viewed it as redundant and less convenient to use than their (usually paper-based) calendar. A few participants had calendars in their kitchens that included the schedule of all their activities, rather than merely medical events. These calendars more easily allowed them to check for scheduling conflicts. One participant, already entering activities into two electronic calendars (Microsoft Outlook and Quicken), explained that updating a third calendar would be burdensome. One beneficiary suggested the online Health Calendar was less convenient because the beneficiary logs into the Internet only once a day. Another suggested that this function could increase its value if the PHR sent beneficiaries alerts or reminders ahead of appointments.

"It's better than the pieces of paper in the drawer."

Beneficiary

Claims.  The Claims page provided beneficiaries with a great deal of useful information. Some beneficiaries were familiar with MyMedicare.gov and used the Website to view information similar to that offered in the PHR Claims page. Beneficiaries were glad to see the information from the MyMedicare.gov site included on the Claims page. However, this page also evoked some questions that were similar to those that arose about other pages (e.g., unknown health providers or facilities), as well as some financial questions. For instance, the amount listed in the “Total Billed” column on the Claims page was at times smaller than the amount in the “Total Paid” column.

Authorized Representatives.  Few participants had used the Authorized Representative page prior to the user observation. Many indicated that they would add their children as authorized representatives to the MyPHRSC tool when they returned home. One beneficiary mentioned being willing to give access to a son; however, because the son would not access the PHR frequently, and with password changes every few months, the beneficiary wondered how useful this would be. Others mentioned that they would be willing to give access to their health providers if the providers thought it would be valuable to them.

"I probably wouldn't go into a password protected site to search for information."

Beneficiary

Healthwise and Trusted Websites.  When beneficiaries first viewed the Healthwise page, they thought it could provide a useful resource. However, the searching process was difficult for some. For instance, when a beneficiary clicked on a letter to view a list of all topics beginning with “H,” a “content error” message appeared. After trying “hypertension,” a long list appeared with everything from “Pregnancy and Childbirth” to “Pseudotumor Cerebri.” Another beneficiary tried searching for “Compression Fracture — Spine" and received no results. When the search was altered to “compression fracture,” a long list appeared, but nothing related to spinal injuries. In general, beneficiaries felt that searching for information using other online resources such as Google.com was more useful and convenient.

MyPHRSC also includes a Trusted Websites page that features links to relevant sites. These links are organized in categories such as “Men’s/Women’s Health”, “Disease Specific Resources”, “South Carolina Resources” and “Social Security”. Some participants thought the list of Trusted Websites was useful and had visited some of the pages through other sources. One thought it was good to have a list of websites that had been vetted, so beneficiaries could be assured that the information on those sites would be useful and not commercial.

Health Assessment: My Family History and My Social History.  As a whole, the Health Assessment functionality appeared to be user-friendly. Most participants had either completed or thought that completing the My Family History section would be useful. One participant was disappointed that the option for listing relatives’ conditions did not include a category related to heart disease. The page did not allow users to input additional conditions other than the fewer than twenty options given. Few beneficiaries had used the My Social History section. One participant was not impressed with the half dozen pre-set questions that it offered (related to tobacco, alcohol, and drug use, seat belt use, and exercise) and wished that there were space to enter additional information beneficiaries considered relevant to their social history.

Emotional Affinity.

In order to discern what, if any, emotional reaction MyPHRSC provoked among beneficiaries, we used an activity allowing participants to talk about some of the emotions they experience when using the tool. We used nine labeled images of common interactions to obtain reactions from users on which pictures evoked feelings that most closely—and least closely—related to how they felt about MyPHRSC. A full display of these images is included in Appendix 2.     

The image of a library was most commonly cited as relating to the experience of using MyPHRSC, because both provide a “wealth of knowledge.” Two beneficiaries identified MyPHRSC as evoking similar feelings to those of being inside a hardware store, although individually the feelings were different. One thought that seeing all the interesting tools was a positive experience while the other felt being in a hardware store was a confusing, mildly uncomfortable situation, similar to the feeling of not knowing how to enter information or of seeing unfamiliar medical terminology. A couple of participants indicated that using the PHR is not similar to being in a convenience store because it does not include everything they would like to see. However, participants did not identify the most negative pictures with MyPHRSC: the frustration of being stuck in traffic or the pressure and nervousness evoked by being pulled over by a policeman.

Integration of MyPHRSC into Beneficiaries' Health Information Management

Overall, it did not appear that MyPHRSC had a great effect on how beneficiaries manage their health information. Primarily, the PHR seemed to be viewed as a source of supplemental information and not as a replacement for beneficiaries’ current health records. For instance, while the Wallet Card replaced a couple of participants’ previous lists, others—even the most tech-savvy—preferred to maintain their old system because it was what they had found could work for them.

Beneficiaries were able to see situations in which MyPHRSC would become more integrated into their health information management. Most of the participants appeared to be permanently settled in the Columbia area, but one indicated that if planning to move, the PHR would help with the transition to a new doctor. There were important examples of the PHR influencing beneficiaries’ interactions with their doctors. One beneficiary experienced seeing a new doctor whose information conflicted with a previous provider about the appropriate treatment for the diagnosis. Upon questioning the new physician about the difference, the physician asked the beneficiary to describe everything that had happened in relation to the diagnosis. The beneficiary used MyPHRSC to develop a chronological list of everything that occurred in relation to this diagnosis.

Most Useful Functions in MyPHRSC. Users perceived greater value in some MyPHRSC functions than others. In order of preference, the Wallet Card, Health Record Summary and Claims history features were identified as being the most useful features. Beneficiaries appreciated the portability of the Wallet Card, perceiving it as a particularly convenient way to share information with providers and touting the importance of having the emergency contact information at hand. The Wallet Card’s tremendous potential to improve the quality of communication between participants and their providers as well as enhancing the doctor-patient relationship was also mentioned.

The Claims feature of MyPHRSC was cited as a useful tool to help beneficiaries keep track of finances. It also gave some details helpful to beneficiaries, such as the name of the provider associated with the claim and the status of the claim. The Health Record Summary was also perceived as useful. Beneficiaries were impressed with the thoroughness and the extent of detail available with the summary.

In terms of other MyPHRSC features, the Medications component of MyPHRSC was reported by beneficiaries as a highly desirable function of the PHR. Beneficiaries reported that the current Medications page was cumbersome to use, but that tracking medications through the PHR is highly desirable for helping beneficiaries to reconcile their medications over time.

Least Useful Functions in MyPHRSC.  The health calendar feature was cited as being less useful than some of the other MyPHRSC features. Beneficiaries found the calendar to be redundant and less accessible than paper calendars such as those that might easily be seen on a refrigerator and showing more than health-related appointments.

Improved Patient-Provider Communication. The PHR’s potential to facilitate dialogue with providers was highlighted by several beneficiaries. The Wallet Card in particular was reported to have enhanced communication between beneficiaries and their providers. The card contributed to improving the quality of dialogue by bringing the beneficiaries’ attention to issues in need of discussion with their provider. This provided the beneficiaries with an opportunity to further engage in their healthcare.

Issues and Concerns.

Lack of Persistence of Certain Information. Beneficiaries observed that the wallet card did not save all previously-entered information. This necessitated that users re-enter some of the information each time they wanted to create a card. Beneficiaries found this to be inconvenient and would rather the PHR allowed them to simply edit the previously entered information.

Difficulty Understanding Medical Terms.  Participants reported that MyPHRSC used some terminology that was not easily understandable. For instance, the use of terms such as “PCP” and “provider” were confusing. Additionally, participants did not understand the SNOMED and ICD codes which were included in the PHR.

Data Inaccuracies. Although the beneficiaries generally trusted the information in the PHR and believed the quality of the claims information provided to be satisfactory, some did express minor concerns regarding the accuracy of the claims information. It was noted that some of the information was inaccurate and that some claims data was missing. Some also questioned the quality of the data that was auto-populated in MyPHRSC due to clinicians and facilities occasionally being listed multiple times and some self-entered information not being retained.

Perceptions around Privacy and Security.  In terms of privacy and security, most patients reported a willingness to share their information in a PHR if it would benefit their care. The beneficiaries did not have significant concerns regarding the integrity of their information in MyPHRSC, however, a few expressed serious concerns about health information being disclosed and used against them by insurance companies or the government.

Discussion Group with Providers

Discussion topics for the meeting with providers covered provider experiences using health information technology, provider understanding of PHRs, challenges and enablers using PHRs, perceived impact of PHRs to provider workflow and suggestions for future PHR development.  Below we provide a summary of key results and preliminary conclusions.

The results of the pre-discussion screener indicate that participating providers had a broad range of experience using health information technology.  Three providers knew about MyPHRSC prior to November; three providers reported using EMRs in their practices; and one provider reported having patients who used PHRs.  Providers who participated came from a variety of practice types in locations throughout South Carolina, with most providers working in family practice and internal medicine, as well as one chiropractor and one optometrist.  Provider experience varied from 12-38 years in practice; with the majority of providers practicing for 18 years or longer.  The results of the pre-discussion screener are included in the Appendix 11.

Provider Experiences Using Health Information Technology.

Overall Health Information Technology Experience.  Providers were asked to what extent they had experience using various forms of health information technology in their practices, as well as what experiences their patients had using PHRs.  The majority of the group reported using some form of electronic technology as a regular part of their practice.  One provider noted that his practice uses Medfusion for on-line pre-registration, conducts virtual online visits, and uses secure email with patients.  However, the general level of sophistication of the technologies which providers used varied widely and a few providers preferred to use more traditional methods of record keeping and communicating with patients such paper medical records and telephone calls. 

Experience Using EMRs and PHRs.  A few providers reported past or current experience using EMRs in their practice, while a couple of providers had patients who used PHRs.  None of the group had interacted with a patient’s PHR.  Providers, however, reported sharing results from the EMR with patients if the patients requested particular items or if they felt it would help patients better understand their conditions or results.

Communication with Patients. Providers discussed their various preferences for communicating with their patients.  A few providers reported that they preferred to use the telephone to communicate with patients, while some preferred online communication via email.  Providers varied in their opinions regarding the type of information they were willing to share over email. Some providers answered non-emergency patient questions by email while others preferred email for sending patient reminders for upcoming appointments or sending patients actual test results.  One participant noted that sending test results via personal email resolved the issues around attempting to reach a patient by phone if that person is not at home.  Email was also viewed by some as a more secure mechanism for delivery of personal health information than telephone voicemail messages to patients’ homes, as this information could be shared with the wrong person.

Provider Perceptions of PHRs.

Providers generally viewed PHRs as a patient-initiated and patient-maintained summary of a patient’s medical and health information from all of the patient’s providers.  Several providers believed PHRs were tools that allowed patients to store their health data in one place and to share this with whomever they designated. 

Patient Self-management of Care and Sharing of Health Information. In the PHR user discussions conducted for this study, NORC found that beneficiaries using MyPHRSC were interested in sharing content populated in MyPHRSC regarding their health, treatment and conditions. Some patients found value in the potential to keep their paper records at home while sharing the information electronically with their doctors.  We asked providers about the kinds of personal records or information patients had shared with them, such as paper records, and how useful this was.  Providers reported that patients had shared their blood sugar frequency lists, blood pressure measurements taken at home and comprehensive military records (TRICARE). 

A few providers noted that when patients share their own personal records it saves time at the actual visit, allowing the doctor to focus on a patient’s needs.  Providers also indicated patients’ management of their own care would help reinforce the provider’s recommendations for treatment, enhancing their trust in the relationship.  A few providers expressed that the patient should ultimately trust the word of the provider and saw little utility in having access to patient information via their PHR.

Providers also noted that the elderly population has a greater need to be responsible for their own care, but they are less able to accurately recall health events and reliably maintain treatment and care at  home.  One provider indicated that patient records are often based on patients’ memories and thus the information shared is subject to recency and primacy effects (the most recent health events remembered or the most salient health events).  On the whole, providers viewed patients’ sharing health information positively and perceived PHRs as a patient tool to enhance management of their own care.

Privacy and Security.  Providers expressed concerns with the security of personal health information and indicated that, if stored in an electronic form, there was a higher likelihood that patient information would be accessed by insurance companies and the government.  Concerns also included the use of online personal health information for data mining and public health purposes without proper consent.  One provider shared a recent news story which indicated that Google was able to evaluate trends in influenza in the U.S. within the past 5 years in advance of the CDC because they mined all the inquiries they received.  The group remained divided as to whether or not the benefits of this public utility outweighed the possible cost of data mining.

PHR Utility.

Overall Utility for Patients.  Providers agreed that patients benefit from maintaining their own health records, and that being in charge of their own PHR could help facilitate their care.  Additionally, providers noted that patients who are on vacation and need quick access to records could benefit from having a PHR. Some participants indicated that the patients who currently maintain paper records and are significantly invested in their own health and care are the same patients who would use a PHR.  A couple providers agreed that roughly 20 percent of their patients would be able to use a PHR, as they anticipated disability would inhibit the rest of their patients from being able to use a PHR. They also agreed that of those patients who would be able to use a PHR, only 5 percent or less would actually use the PHR.  Overall, providers agreed that patients could find PHRs helpful, but the patients who need them the most may also be the least likely to adopt.

Overall Utility for Providers.  While PHRs could be valuable for patients, providers noted that PHRs could not take the place of EMRs in a provider’s office.  While not a substitute, providers suggested a practice without EMRs could still benefit from having access to patient PHRs.  One provider indicated that the PHRs would be more helpful to providers in an Emergency Room particularly when dealing with patients who are not able to provide a medical history, for example, if they were unconscious or were suffering from dementia.

Control and Access to PHR.  While providers articulated numerous situations in which PHRs could help them or their patients facilitate better care, providers also identified key criteria which would be necessary for PHRs to be useful for them.  Firstly, providers suggested PHRs must be owned and controlled chiefly by patients for them to be effective.  Providers indicated they do not have the time nor do they receive any compensation for using PHRs. While PHRs could be useful for public health purposes, for example, to data mine for health trends or disease occurrences, the group generally agreed that consumers’ personal health information should not be controlled by or available to the government without consent. 

PHR Content.  Although previous research suggests providers would prefer not to include content from EMRs such as clinical information or provider notes[34]^,[35], several providers in this group suggested they already share data directly from the EMR with patients.  One provider indicated that he regularly shares EMR information with patients to facilitate their care, and this information would be useful to include in the PHR as it could facilitate their care in other settings.  Another participant added that there were circumstances when sensitive information was not included in the medical record.  A few providers agreed they would be willing to share the EMR data with patients if it were useful, but the best strategy would be to allow patients the flexibility to decide what they want included in the PHR.

Reimbursement.  Consistent with findings from previous research on provider perceptions of PHRs, providers reported that reimbursement will affect their decision to use PHRs.  PHRs might require additional time and effort, and many providers expressed feeling overextended already.  Participants indicated that less than 2% of medical school graduates choose to enter primary care today because they believe reimbursement is not adequate.  In general, providers agreed that they already perform a number of tasks that they are not compensated for and would, therefore, be reluctant to use a PHR.

Data Accuracy and Relevancy of a Claims-Based PHR.  Many providers indicated that claims data can be inaccurate or incomplete. For example, when patients obtain covered prescriptions from pharmacies like Wal-Mart or fill other medications from their own out-of-pocket or health savings accounts. Thus, providers discussed the issue that claims information only provides one view into the patient’s health care records, and, is likely incomplete, making it less useful for providers.  One provider noted reviewing patients’ reports from the insurance companies before patients come in for visits to assist in providing care, filtering the data that is irrelevant and understanding its limitations.  Overall, providers believe that the PHR should provide an accurate picture of the patient’s health through accurate and comprehensive data, and that claims information alone does not achieve this goal.

Analysis of Usage Data

This analysis of the usage data provided quantitative information that contributed to answering the key research questions of the evaluation. This section provides descriptive statistics about variables of interest, and then presents multivariate results for the three study hypotheses.

We provide an overview of the total South Carolina Medicare population, the population targeted for the purposes of this pilot, and the users registering for MyPHRSC as well as those that registered and logged in more than once into the tool. In the following section, we describe the characteristics of the registrants and users.

Figure 1.    Relationship between South Carolina Medicare Beneficiaries, Registrants, and Users in the MyPHRSC Pilot. 

Figure 1 provides an overall snapshot of the number of South Carolina Medicare beneficiaries who took advantage of the MyPHRSC tool.

User Demographics.

Fifty-three percent of MyPHRSC registrants were male and 47 percent were female. As of 2007, 41 percent of Medicare enrollees in South Carolina were men and 59 percent women,[36] suggesting that men were disproportionately likely to register for MyPHRSC.  A chi-square goodness-of-fit test confirms that this gender composition of registrants is significantly different than what would be expected in South Carolina (p<0.01). 

The age range of registrants was 26 years to 99 years of age, with the majority of beneficiaries between 65 years and 74 years of age (Figure 2). Twelve percent of registrants were under the age of 65, and 26 percent are over age 75.  Individuals aged 65-74 were more likely to register for MyPHRSC and individuals under 65 or over 75 were less likely to register for the PHR than would have been expected from the age distribution of Medicare beneficiaries in South Carolina in 2007.[37]  A chi-square goodness-of-fit test confirms that the ages of MyPHRSC registrants differ significantly from those of the South Carolina Medicare population.

Figure 2.    Frequencies Of MyPHRSC Registrants by Age Compared with All South Carolina Medicare Beneficiaries.

Figure 2 shows he age range of registrants was 26 years to 99 years of age, with the majority of beneficiaries between 65 years and 74 years of age

Illness and Condition Diagnoses.

Among the 4,114 MyPHRSC registrants, there were 972 unique illness/condition diagnoses in the MyPHRSC data set.  The ten most common diagnoses were: metabolic disorder, high blood pressure, cataract, skin lesion, disorder of soft tissue, joint pain, osteoarthritis, disorder of back, chest pain, and disorder of esophagus.  Because so many unique conditions, illnesses, and symptoms existed among the beneficiaries in the sample, NORC, in conjunction with ASPE, selected eight chronic conditions for closer examination.  The selected conditions were: high blood pressure, coronary artery disease, heart failure, diabetes, chronic obstructive lung disease, asthma, cataract and osteoarthritis.  These eight conditions were selected because of their prevalence in the Medicare population. A 2007 Medicare Payment Advisory Commission (MedPAC) report cited high blood pressure, coronary artery disease, heart failure, diabetes and chronic obstructive lung disease as among the most prevalent conditions within the Medicare population[38].  Two conditions--cataract and osteoarthritis--were added based on a list developed by CMS of conditions that cause high health burdens and represent the highest cost for Medicare[39].  There was also effort to coordinate the selected conditions with those chosen for the managed care PHR evaluation, resulting in the inclusion of asthma in our analyses. Among the eight selected conditions, high blood pressure was the most common diagnosis for MyPHRSC registrants.

 Figure 3.      Percent of MyPHRSC Registrants with a Priority Condition

Figure 3 provides information on the overall prevalence of the eight chronic conditions among registrants.

Most Accessed Functions.

The Health Record Summary, or Continuity of Care Record (CCR), was accessed by 2,773 unique users, representing over 90 percent of all beneficiaries who ever logged in to MyPHRSC.  The active medications, medication profile, visit summary, summary of current illnesses, and claims information pages were all accessed by more than one-third of all beneficiaries who ever logged in. Other highly viewed pages include the Provider Detail, Social History and Health Calendar.

Figure 4.    Most Popular MyPHRSC Pages

Figure 4 shows the seven most popular pages of MyPHRSC.

Least Used Functions.

Several pages in the PHR were only accessed a handful of times over the course of the evaluation. These included the visit void page, the medical procedures detail page and the health journal edit page. These pages were not major features of the PHR and generally tended to be voiding and editing features.

Usage Over Time.

As noted earlier in the methods section, two dependent variables were considered.  The first dependent variable categorized registrants by a threshold of total number of logins:  never, once, or more than once.  The second dependent variable was the total number of months during the pilot in which registrants (i.e., users) logged in to MyPHRSC.  More than half of all registrants only logged in to MyPHRSC during one calendar month of the pilot.  Although these users may have logged in more than once during that single month, for the purpose of examining usage over time, one or more logins in a particular calendar month is reported simply as activity during that month.  Only 21 percent of users (n=852) accessed MyPHRSC in at least two separate months of the pilot. 

Figure 5.    Number of Users Logging In During Specified Number of Unique Months

Figure 5 displays the number of users who logged in at least once during each number of months.  A small number of users accessed MyPHRSC in 6 or more separate months, but no single user accessed MyPHRSC in all 12 months of the study. 

Hypotheses.

HYPOTHESIS 1: BENEFICIARIES WITH CHRONIC CONDITIONS OR DIAGNOSES ARE MORE LIKELY TO USE MYPHRSC THAN BENEFICIARIES WITHOUT SUCH CONDITIONS.

Ever log in. We did not find consistent patterns of use based on illness diagnosis.  Individuals with the diagnosis of “coronary artery disease” and "cataract" were more likely than individuals without those diagnoses to return to MyPHRSC and log in after initial registration (p<0.05).  Individuals with the diagnosis of “diabetes” were significantly less likely (p<0.05) than individuals without diabetes to return to MyPHRSC and log in after initial registration.  None of the remaining selected chronic conditions were statistically significant predictors of logging in to MyPHRSC after registering.

Log in more than once. Individuals with the diagnosis of "cataract" were 1.2 times more likely to log into MyPHRSC multiple times than individuals without this diagnosis.  Individuals with a diagnosis of "osteoarthritis" were significantly less likely (0.8 times, p<0.05) to log in more than once than individuals without osteoarthritis.  No significant difference existed between individuals having no chronic condition and those having one or more chronic conditions. 

Log in more than two separate months. Individuals with the diagnosis of “chronic obstructive lung disease” and "cataract" were significantly more likely to log in to MyPHRSC in 3 or more separate months than individuals without this condition.  These individuals were 1.4 times (p<0.05) as likely to have logged in during 3 or more separate months during the study period. In terms of logging in during multiple months, no significant difference exists between individuals having no chronic condition and individuals who have one or more chronic conditions.     

HYPOTHESIS 2: WOMEN ARE MORE LIKELY THAN MEN TO USE MYPHRSC.

Ever log in. In multivariate analyses, we found that women were not more likely to use MyPHRSC than men.  Rather, we found that men were significantly more likely to return to MyPHRSC and log in after initial registration.  Men were approximately 1.5 times (p<.01) more likely than women to ever log in.

Log in more than once or in more than two separate months. Women were no more likely than men to log in to MyPHRSC more than once. Women were no more likely than men to log in to MyPHRSC in multiple months of the pilot.

HYPOTHESIS 3: YOUNGER BENEFICIARIES ARE MORE LIKELY TO USE MYPHRSC THAN OLDER BENEFICIARIES.

Ever log in. The youngest beneficiaries (aged 64 and under) were not significantly more likely to use MyPHRSC than their older counterparts.  In fact, individuals between the ages of 65 and 69 were slightly more likely to return to MyPHRSC and log in after initial registration.  Statistical significance was found in comparisons with two of the four other age groups (i.e., 64 and under; 75 — 79).

Log in more than once. Beneficiaries aged 65-69 were less likely to log in to MyPHRSC more than once than any other age group.  This finding was statistically significant (p<0.05).

Log in more than two separate months. Users under the age of 65 were less likely to use MyPHRSC for more than two months than every other age group, with statistical significance in all comparisons except with users over age 80.  

Discussion

Findings from this project, although limited in scale, provide valuable insights into specific needs of Medicare beneficiaries with respect to PHRs, which can inform future studies. Below, we provide a discussion of findings from qualitative and quantitative evaluation activities categorized by key themes. As appropriate, findings from the environmental scan are included. Findings are organized in the areas of general findings, determinants of MyPHRSC use, the usability and utility of MyPHRSC and the relationship between MyPHRSC outreach and registration.

General Findings.

Health Care Priorities.  Beneficiaries in this evaluation reported that their most important health care concerns included: continuity of care, reduction of healthcare costs, managing their spouses’ care and self-managing conditions such as diabetes. Beneficiaries also indicated that coordination of care in a fragmented system was an important health care priority for them. It is important to have an understanding of beneficiaries’ health care priorities in other to identify their needs and expectations for PHRs and the functions they desire in PHRs. Certain health information and supporting features and functions may be more relevant to some users than others. 

Health Information Management. Beneficiaries in the discussion groups and user observations revealed that they maintained paper medical records, and one beneficiary preferred this method to using a computer.  Beneficiaries also reported using calendars for appointments, and keeping files for medical records and medical bills.  Some beneficiaries used computer databases to store information on medications or track conditions.

Beneficiaries’ methods for storing health information varied based on the type of information in question. In user observations studies, NORC found that much of the personal health information which beneficiaries kept fell into the just-in-case (when information is kept away but reasonably accessible) category as opposed to the just-because category (when the information is kept in the household but no other storage strategy is assigned because of a temporal relevance). Other information was stored as older records (comingling medical and tax information) in less accessible locations, such as the attic (the just-in-case or just-because categories). Contact information, both for health providers and emergency contacts, fell under the –just-at-hand (when information is stored in a visible or readily available, familiar location) storage strategy and was often kept in very accessible locations. Some participants carried their personal health information, with some participants using the MyPHRSC Wallet Card for that information. Use of this technique, where information was kept with a household member at most times, fell into the just-in-time category.

Data Ownership. Beneficiaries viewed themselves as the primary owners of their own health information, and most had established mechanisms for managing their own care. 

Information Sources. Beneficiaries employed different methods for obtaining medical information. Some participants kept general medical reference books in their office or den. Some also received medical newsletters, although they would only keep articles they found useful and not store all copies of the resource. Many of the paper copies of reference material were stored using the just-because strategy, where no specific storage strategy is used. Beneficiaries who participated in the discussion groups also revealed that they frequently used the Internet to search for health information.

Internet and computer experience.  Results of the environmental scan showed that the elderly are less likely to have experience using computers, access to the Internet, and broadband connection than those under age 65.^[40]  At the same time, an increasing number of older adults are accessing the Internet. In 1996, only 2 percent of adults 65 and older were ‘online’, and by 2004, that number rose to 22 percent.^[41] In general, beneficiaries from the user observations, discussion groups and semi-structured interviews often used the Internet and owned computers at home. Most described themselves as being relatively comfortable using these technologies, although there were a handful of beneficiaries who did not consider themselves computer savvy.  Beneficiary discussion groups revealed that physical difficulty accessing the home computer, such as being unable to climb the stairs when a computer was located on a second floor at home, was an impediment to PHR use.

Privacy and Security. Although discussions with both users and non-users of MyPHRSC revealed some security concerns with the PHR, many suggested that having access to information and being able to share with providers and other care-givers would outweigh the risks. Similar findings emerged in the environmental scan where consumers cited major concerns with the security and confidentiality of information contained in PHRs but suggested that the convenience of access to their information and the ability to share this with others would be more important. [42]

User Demographics. Fifty-three percent of MyPHRSC registrants were male and forty-seven percent were female. On the whole, the Medicare population tends to be female (56 percent)[43] and this trend is also reflected within South Carolina where fifty-nine percent of Medicare enrollees are women.[44] The statistics in this study suggest that men were disproportionately likely to register for MyPHRSC. Therefore, the gender composition of registrants is significantly different from what would be expected for both the general Medicare population and for MyPHRSC users.

Although it was hypothesized that women would be more likely than men to use MyPHRSC, analysis of usage data showed that men were significantly more likely to return to MyPHRSC and log in after initial registration. Additionally, women were no more likely than men to log-in to MyPHRSC more than once or for multiple months of the pilot.

Registrants of MyPHRSC ranged from 26 years to 99 years of age, with the majority of beneficiaries being between 65 and 74 years of age. Individuals aged 65-74 were more likely to register for MyPHRSC and individuals under 65 or over 75 were less likely to register than would be expected from the age distribution of Medicare beneficiaries in South Carolina in 2007. The ages of MyPHRSC registrants, therefore, also differs significantly from those of the South Carolina Medicare population.

PHR users were defined as individuals who logged on to MyPHRSC once or more over the course of the pilot. Contrary to hypothesis, younger beneficiaries (those aged 64 and under) were not significantly more likely to use MyPHRSC than their older counterparts.  In fact, individuals between the ages of 65 and 69 were slightly more likely to return to MyPHRSC and log in after initial registration.  However, members of the 65 to 69 age group were also less likely than others to log in to MyPHRSC more than once. This may suggest that beneficiaries between the ages of 65 and 69 are particularly interested in a personal health record application, but that MyPHRSC did not satisfy their needs. Additionally, it is possible that Medicare beneficiaries in the 65 to 69 age group do not have significant medical illnesses that require constant monitoring and that they are less likely to make use of a tool like a PHR.

Uptake of MyPHRSC. While there are no industry standards on what is considered high uptake of PHRs, the NORC evaluation found that the majority of beneficiaries who used MyPHRSC logged in during only one calendar month of the pilot and over one-quarter of registrants never logged in to MyPHRSC. The latter group may not have received sufficient incentive or information on the value of using MyPHRSC, and the former group may not have seen information or features of sufficient value to encourage repeat use.  In light of the initial 48-hour delay between registration and when a beneficiary can view his or her information in MyPHRSC, follow-up communication by e-mail may be useful in encouraging registrants to return to MyPHRSC and log in, particularly if the message alerts the beneficiary that his or her information is now available. 

There are several reasons that participants might have chosen not to sustain use of the PHR after registering and initially logging in. One reason cited for not using the PHR was that it did not fit into the beneficiary’s regular Internet “workflow”, and this may have been a contributing factor to forgetting to use the PHR. Additionally, it is possible that beneficiaries may not have understood the value of continuing to log in to MyPHRSC to review their claims information over time, did not find the content useful or compelling enough to return or otherwise did not have sufficient incentives to encourage continued use. Discussions with users and nonusers also indicated that reasons for not using MyPHRSC included being too busy or being otherwise healthy. One of the key challenges that also prevented users and nonusers from continuing use of the system was related to remembering log-ins, user IDs and passwords.  Similarly, it was challenging for beneficiaries to have to change their temporary passwords every 90 days.

Illness/Condition Diagnosis. Chronic illness is highly prevalent among members of the Medicare population—especially the elderly. One survey of Medicare beneficiaries indicated that 65 percent of all elderly people had two or more chronic conditions, and 34 percent of seniors reported limitations in mobility or activities of daily living. [45]  Another report found that patients with five or more chronic conditions represented 50 percent of the Medicare population[46]. PHRs can be particularly useful to patients who have serious or chronic conditions[47].

A survey conducted in 2007 by the Foundation for Accountability (FACCT) as part of the Markle Foundation’s Connecting for Health Collaborative found that those with chronic conditions reported the highest interest and most urgent need to use PHRs.[48] These individuals also perhaps have the greatest motivations for using PHRs. Based on this, our study hypothesized that there would be greater use of the PHR among registrants with the aforementioned chronic conditions of high blood pressure, coronary artery disease, heart failure, diabetes, chronic obstructive lung disease, asthma, osteoarthritis and cataracts. Usage data analysis showed that, contrary to the hypotheses, there were no consistent patterns of PHR use based on illness diagnoses. When all beneficiaries with at least one selected chronic condition were pooled together, there was no increase in the likelihood that these beneficiaries would return and log in to MyPHRSC after registering compared to beneficiaries without any of the specified chronic conditions. Beneficiaries diagnosed with high blood pressure, heart failure, chronic obstructive lung disease, or asthma were no more or less likely to return to MyPHRSC and log in after registering than beneficiaries without these conditions. MyPHRSC currently does not include information on laboratory test results and tracking of these results over time. It is possible that if these functions were available we may likely see an increased use of the PHR by patients with chronic conditions. We also note that the study period was only a year and that this may not be sufficient time to draw conclusions on linkages between chronic conditions and PHR use.

Determinants of MyPHRSC Use.

Reasons for Using MyPHRSC. Discussion groups with MyPHRSC users also provided an understanding of some factors influencing beneficiaries’ use of the PHR. Many beneficiaries initially logged into the PHR out of curiosity. Some participants used the PHR to update and print their wallet cards, and a few reported using the PHR to print the healthcare summary. Many participants used the PHR to view their past claims information.  Generally, participants felt that the PHR could be a very useful tool for them.

Reasons for Not Using MyPHRSC. Usage data analysis revealed that 26 percent of registrants to the PHR did not log in later and that more than half of all registrants only logged in during one calendar month of the pilot (although these users may have logged in more than once during that month).  Additionally, although a small number of users accessed MyPHRSC in six or more separate months, no single user accessed MyPHRSC in all twelve months of the study.   

Beneficiaries reported a number of difficulties using the PHR. The most significant issues for beneficiaries were focused around signing in to the site, e.g., user ID or password lockout.  Findings from beneficiary discussion groups indicated that both beneficiary users and nonusers experienced trouble signing in the first time; while a few beneficiaries reported the PHR was easy to log in to.  In user observations, beneficiaries did not have difficulty logging in but did express difficulty with renewing and remembering their passwords.  Some participants required technical assistance on multiple occasions in order to log in successfully to the PHR. Using and remembering their user IDs and passwords also created a significant barrier to logging in to the system for both users and nonusers of the PHR.

Beneficiaries also noted the inaccuracy of some of the claims data in the PHR. At times, some claims data was missing or facilities and clinicians were listed incorrectly or at multiple times. The delay in the updating of claims in the PHR was also a challenge.  Beneficiaries noted that the paper summary of claims information arrived in the mail relatively quickly following a visit or other medical event, and in order for the information in the PHR to be useful for them it would need to also be updated in a timelier manner. All these issues with claims data were a point of concern for some participants and were cited as a significant barrier to use of MyPHRSC. For instance, not having reliable claims data in the PHR limited the utility of beneficiaries’ sharing the information with their doctors.   

Many of the concerns beneficiaries identified appeared to be related to issues arising with entering data into the PHR. This was particularly so with the entering of medication information, a process that was often confusing and frustrating. In the user observation studies, some beneficiaries were unable to enter their medication information without guidance. Additionally, beneficiaries observed that the Wallet Card did not save previously-entered information. This necessitated that users re-enter information each time they wanted to create a card which beneficiaries found to be inconvenient and would rather the PHR allowed them to simply edit the previously entered information.

Provider Perceptions of PHRs. Discussion groups with beneficiaries revealed that they would be more likely to use a PHR that was provider-recommended.  Therefore, it is important to understand providers’ reasons for using and not using PHRs. The discussion group conducted with providers revealed that workflow disruptions, validity of information in PHR, reimbursement issues and concerns about the limits of claims data were all barriers to provider use of PHRs. Other results that corroborated these findings emerged in the literature scan. Tang et al. (2006) cited provider concerns about liability risks, lack of reimbursement or financial incentives to adopt PHRs, increased workload and negative impacts on provider workflow as some of the barriers to provider participation in PHRs.^[49]  Another study similarly found that most providers had concerns about the validity of claims data in the PHR for use in making treatment decisions and found that integrating the PHR into practice workflow was a major barrier to use[50].

On the whole, the discussion group with providers revealed that they viewed patients’ use of PHRs and their sharing health information positively. They considered the PHR to be a valuable tool to help enhance patient understanding of their own care. This said, providers also believed that patients should control access to the PHR and should have the responsibility for deciding what data would ultimately be included in the tool. Additionally, providers reported that they would be willing to share EMR data with PHR applications if it would help improve patient care. 

Enablers.  Although beneficiaries reported having some difficulties with the MyPHRSC tool, they identified some enablers that served to help them overcome the issues they encountered. For instance, many beneficiaries found that the strong technical support provided through e-mail and by telephone using the toll-free number helped them to persevere despite difficulties signing into the PHR.  Furthermore, many found the PHR as a whole was relatively easy to use once they were able to sign in.

Key Benefits. In general, participants thought that the PHR was a helpful tool that provided useful information about their health and indicated that they viewed the tool as a method for improving communication with their providers. For instance, the information on the PHR helped one participant provide necessary information during a transition to a new doctor and helped another discuss an unknown diagnosis with the provider. NORC’s environmental scan revealed that other studies have also found a positive impact of PHRs on users’ relationships with their providers. A survey by Keseleman et al (2007) found that, as consumers kept better track of their own health care, they were better able to inform their doctors at visits, and many reported improvement in relationships with physicians as a result.^[51],[52] Another study found that generally healthy consumers reported forgetting to ask health related questions that they had intended to discuss during provider visits. Providers participating in the discussion group for this evaluation noted that the elderly population had a greater need to be responsible for their own care, but they were less adept at accurately recalling health events and keeping up records. PHRs may help this group to have more productive interactions with their physicians by helping them keep track of topics for discussion.

Beneficiaries detected additional utilities in MyPHRSC as well. Many were impressed with the tool’s thoroughness, and they expressed positive feelings about it, describing its similarity to a library because of the many resources it provides. The Wallet Card feature was applauded for its portability — it could easily be taken along to their clinician visit and could provide contact information in case of an emergency.

Overall, beneficiaries considered the PHR a source of supplemental health information rather than a replacement for their current health records and used it as just one component of their greater information management strategy. For instance, while the Wallet Card replaced a couple participants’ previous lists of medications, conditions or key contacts they had kept, others—even the most tech-savvy—preferred to maintain their old paper system that they had established already worked for them.

MyPHRSC Utility.

Highly Used Functions.  For this evaluation, utility is defined as the quality or condition of being useful in reference to the functionality of the PHR’s design.  Findings of NORC’s environmental scan show that researchers have suggested that the number of times certain data elements or functions within the PHR are accessed can serve as a measure of utility.  Identifying the highly used functions of the PHR can also provide an understanding of which features users found most valuable and ultimately provide insight into the features and functions that encourage PHR adoption by consumers. The Health Record Summary and Claims functions were seen as useful. Indeed, other than the PHR’s Home page, the Health Record Summary saw more views than any other pages. Beneficiaries were impressed with the thoroughness and the extent of detail available with the health record summary, feeling that it was far more comprehensive than paper summaries. The claims feature of MyPHRSC was cited as a useful tool to help beneficiaries keep track of finances. It also gave some details that were helpful to beneficiaries such as the name of the provider associated with the claim and the status of the claim.

Beneficiaries also reported that, although the current Medications page was cumbersome to use, tracking medications through the PHR was highly desirable for helping beneficiaries to reconcile their medications over time. Analysis of usage data revealed this page to be the third most used feature within MyPHRSC.

That a particular page was not among those most accessed in the PHR does not necessarily indicate that the feature was of little use to beneficiaries.  It is possible that the feature might be of significant value to PHR users but that it is only occasionally needed. In the case of the Wallet Card, for example, usage data analysis revealed that this feature was infrequently accessed by beneficiaries. However, the wallet card feature was also highly favored among beneficiaries in the user observations and was the function which beneficiaries most commonly cited as useful for them. Beneficiaries liked the ability to print the card and bring it along to their appointments, and thought it would be especially important to have at hand while traveling. 

Least Used Functions.  The calendar feature was one of the least popular among beneficiaries. Many beneficiaries reported maintaining paper calendars at home, and preferred using these methods to an online calendar. They found the calendar to be redundant and less accessible than their paper calendars. Additionally, beneficiaries had difficulties creating multiple entries on the calendar.

MyPHRSC Usability.

Using the PHR.  In order to entice older individuals to use PHRs, the tool must be designed to accommodate their needs. For this evaluation, usability is defined as a quality attribute that assesses how easy user interfaces are to use, and also refers to the methods for improving ease-of-use during the design process.  Understanding beneficiaries’ experiences in utilizing MyPHRSC and identifying what beneficiaries appreciate in the PHR sheds light onto the factors that must be taken into account in developing a tool that is valuable to the population. Most beneficiaries agreed that the PHR was easy to use once they were able to log in.  However, beneficiaries gave many suggestions for improving the tool and making it more accessible for them. Overall, beneficiaries would have liked the text in the log-in screen to have a larger font and for the log-in text boxes to be located more centrally on the page.  Additionally, participants in beneficiary discussion groups stated that having the ability to change the font size within the PHR helped make the PHR easier for them to read.

However, beneficiaries reported having some difficult with data entry, particularly with the Medications page of the PHR. The user observations study revealed that the confusion centered on figuring out how to enter the information, search for a drug they may be taking, select the route, and identify refills. User observations also suggested that the language used throughout the PHR was sometimes too technical for the beneficiaries to understand. Beneficiaries sometimes blamed themselves for not being able to easily navigate the system and viewed difficulties using the tool as their own misjudgment.

Although they were impressed by the extensive information in the Health Record Summary, beneficiaries in user observations were overwhelmed by the large list of illnesses and conditions and also some incorrect information on the page. The page also included information, such as medical terms, that were too technical for beneficiaries to understand. The environmental scan showed that, according to the results of the 2003 National Assessment of Adult Literacy, many individuals have difficulty reading and comprehending health information and that adults age 65 and older have lower average health literacy levels than adults in younger age groups.  It is therefore necessary to present health information in a manner that is easily understandable to users. 

Additionally, user observations studies indicated that beneficiaries were using the PHR but that they were not aware of all the functions offered by the PHR. Beneficiaries also favored the inclusion of certain types of additional data into the PHR.

Outreach and Registration.

In order for PHRs to gain widespread adoption, consumers must be made aware of the availability and advantages of using this tool, and they must be taught how to use them.  After all, at the most basic level, an individual’s usage of PHRs depends on knowing what PHRs are and how they operate.  Although there were two major events that may have resulted in temporary spikes in beneficiary registration for MyPHRSC, many subsequent large events which included audiences of more than 100 did not correspond to spikes in registration. However, two small workshops with audiences of 19 and 35 seem to be linked to small increases beyond the average daily registration.  This suggests that smaller events, or workshops where beneficiaries are engaged one-on-one, may be more effective in encouraging registration. 

On the whole, there is insufficient data to determine the effectiveness of outreach activities on beneficiary registration for MyPHRSC.  While several specific outreach events appear to be linked to an uptake in beneficiary registrations on a particular day, there were substantially more outreach events that did not seem to be linked to any increase in registration for MyPHRSC.  It is possible that the outreach events linked to increased registrations on a particular day had beneficiaries register at the time of the event, while other outreach events may have relied on beneficiaries to register on their own at home at a later time.

The use of electronic or paper-based prompts, which were not a component of MyPHRSC outreach efforts, may help to remind beneficiaries about the tool and to encourage further use. Additionally, a referral from their physician, coupled with follow through communications (either face-to-face or by e-mail) may be likely to improve use of the tool.  It is also important to consider the potential impact of “viral marketing,” or marketing by word-of-mouth.  Referrals from friends, families, or providers may not be readily apparent in overall registration numbers as they can occur at any time, and are not easily identified as being tied to a specific outreach event. Future efforts to attempt to link outreach activities to beneficiary registration for MyPHRSC could be improved by requesting that beneficiaries report a referral source when registering.

Results of the data analysis also suggest that, while generating awareness of the existence of PHRs is certainly a critical first step, education alone is not adequate to ensure that consumers will adopt PHRs and use them on a regular basis. 

MyPHRSC Usage Over Time. The majority of beneficiaries who used MyPHRSC logged in during only one calendar month of the pilot.  While a substantial number of users logged in during two or three separate months, few users logged in during four or more separate months of the pilot.  Beneficiaries may not understand the value of continuing to log in to MyPHRSC to review their claims information over time, or may not find the content useful or compelling enough to continue to return to MyPHRSC after viewing it once or a handful of times.  Email messages or other reminders may be helpful in encouraging repeat use of MyPHRSC.  Such reminders could coincide with the addition of new information to MyPHRSC or could be generated if a specified time interval has passed without a beneficiary logging in.

In examining the general trends of MyPHRSC usage over time, it is clear that beneficiary use of the PHR drops off one or two months after they first registered. Experts have suggested repeated or regular use of PHRs as a key measure of the usefulness of the tool. At the same time, even a one-time visit to the PHR may signify usefulness and the decreased use of MyPHRSC over time do not necessarily mean that beneficiaries are not getting value from the tool. It may instead indicate that beneficiaries may have received the needed value during their initial use of the PHR and therefore do not have a need to return during the rest of that year. However, for many consumers, entering the data routinely and consistently, and printing it for future use (such as sharing with providers) serves a greater purpose.^[53] 

Conclusions and Recommendations

The findings from the evaluation of the FFS PHR pilot provide valuable information to the PHR community on what is needed to design PHRs that are both useful and easy to use for Medicare beneficiaries.  Results also offer policymakers, consumers and providers the benefit of understanding the particular role that PHRs can play in the lives of Medicare beneficiaries.  The following conclusions elaborate on considerations related to the usability and utility of PHRs for beneficiaries, recommendations on usage data to track and suggestions for future areas of study. 

Usability of PHRs for Beneficiaries.

Automatic Population of Data. In both the discussion groups and observation activities, beneficiaries consistently reported that the more data included in the PHR, the better.  Although the population disagreed on the end limit of how much data was enough (e.g. whether beneficiaries would want to see MRIs even if they could not interpret them), the overwhelming consensus was to include as much data as possible in as clear a manner as possible.  Similarly, entering data into the PHR proved difficult, as some beneficiaries were not computer-savvy, some of the terminology in the PHR was unfamiliar and the method for entering data was at times complicated. Beneficiaries found that it was time consuming and frustrating to enter the data themselves, and although some persevered and entered their data because they perceived the value of having access to the information to be greater than the loss of having to enter it, all beneficiaries would benefit from having more data pre-populated in the PHR and less self-entry.

Simple Login Process Necessary.  A significant barrier to beneficiary use of PHRs was successful login. Many beneficiaries were not able to log in without assistance the first time, and many reported difficulties logging in each time.  A login name and password that beneficiaries generate themselves would help beneficiaries to remember these items and enter them correctly.  Additionally, while security and privacy were important to beneficiaries, they would prefer being able to keep their passwords for a longer period of time, if not permanently, rather than replacing the passwords every 90 days for security.

Address Inaccurate or Missing Data.  Future claims-based PHR tools will need to be developed in consideration of the claims data process. While some claims are submitted on time and are free of error, many are submitted late and include inaccurate diagnosis codes. Some beneficiaries were confused by some of their diagnoses, while others did not see any data in the PHR because their claims had not yet been submitted.  In order to ensure data in the PHR are accurate and is provided to beneficiaries in a timely manner, PHR developers should consider importing clinical data as an alternative to claims data. Because of the complicated diagnosis coding language used for medical billing and as a result of the fact that physicians may code things differently, claims data are highly prone to inaccuracies[54]^,[55]. For instance, claims can contain incorrect, incomplete or outdated diagnosis and provider information. As a result, many physicians have raised concerns about the use of such data in PHRs because of the potential serious ramifications of acting on imprecise information[56]^,[57].

Strong Technical Support. A strong technical support team is instrumental for beneficiaries to be able to easily use the PHR. Therefore, such technical support should be included up-front as well as on an as-needed ongoing basis. Many beneficiaries in this pilot depended upon the telephone support of QSSI to guide them through logging in and navigating the system. Perseverance in using PHRs despite the evident barriers may be a factor of the level and dedication of technical support services offered.  Although not all PHR vendors will be able to provide this support, NORC recommends developing PHRs using usability design and considering the end user first and foremost.

Utility of PHRs for Beneficiaries.

PHRs provide the most utility for beneficiaries when they include features and functions that facilitate patient-provider communication, and provide a means for coordinating care across providers.  The most used and most liked functions in MyPHRSC included printing the Wallet Card, as beneficiaries saw this as a mechanism to share information in an emergency situation on travel, and the Health Record Summary, which provided the most comprehensive information about beneficiaries’ health and care over a 2 year period. 

Inclusion of other functions such as a medication history, laboratory test results, clinical decision support, enhanced search, context sensitive help and secure messaging with providers, will all enhance PHR utility for beneficiaries. Additionally, beneficiaries require tools to manage chronic illness, and functions such as tracking for conditions, reminders for conditions tracking, medications and appointments related to conditions, and graphing of vitals and lab results will all support beneficiary management of care. User observations and discussion groups with beneficiaries also revealed that they have different desires and needs for PHRs. Thus, in order to effectively address the different needs, it might be necessary to offer a PHR which consists of separate modules rather than implementing a one-size-fits-all system.

E-mail messages and reminders that coincide with the addition of new information to MyPHRSC would also be helpful. These reminders can be instrumental in encouraging repeated use of MyPHRSC as they could be generated if a specified time interval has passed without a beneficiary logging in. Many beneficiaries also reported having supplemental insurance and stated that it would be of great benefit to be able to view health information from multiple payor and provider sources in the PHR rather than merely those originating from the Medicare system.

Future Areas of Study.

The evaluation of Medicare FFS beneficiaries identifies critical areas for future study of PHRs and how these can be designed to meet the needs of Medicare beneficiaries.  This evaluation revealed that PHR utility is in the ability to share the PHR information with providers and the ability to coordinate care across all providers.  Beneficiaries often deal with multiple conditions and medications across multiple providers, and discussion groups with and observations of beneficiary MyPHRSC users revealed that beneficiaries do not have a single place to store all of their important health information and track their care.  Thus, appointments may be missed and medications may go unfilled, contributing to an increased cost and fragmentation of the health care system.  Further evaluations should investigate how PHRs can be used to coordinate patient care across the continuum of care. 

Integration into Beneficiary Workflows. While PHRs may be developed to coordinate care for beneficiaries, it is likely that beneficiaries will not use the PHRs if they are not designed with usability in mind. It is essential to consider how to integrate PHRs into beneficiary workflows. For example, many beneficiaries have become accustomed to writing appointments on a calendar they store on their refrigerators which includes all of their other appointments.  A PHR designed to coordinate care to fit into beneficiaries’ workflow might incorporate an online calendar that can be downloaded to a computer desktop or printed, and can include the ability to add all non-health related appointments.

Outreach Activities and PHR Use. Outreach activities will be an important component of utilization analysis for future investigations, as this study was unable to define the linkages between outreach and PHR use.  Adoption of PHRs by beneficiaries will require effective outreach mechanisms. 

Chronic Conditions and PHR Use. Additionally, utilization analysis in further studies should systematically study the incidence of chronic conditions and PHR use.  While this study was able to identify the frequency of use and most and least used PHR functions, further research is needed to determine which PHR functions are most beneficial for which subpopulations of beneficiaries, including which types of chronic conditions require which functionality. This issue will be addressed in the evaluation of the Medicare PHR Choice pilot, a new program offering one of four PHRs to Medicare beneficiaries in Utah and Arizona. As part of this evaluation, the outcomes of PHR users with chronic conditions will be compared to nonusers who have the same disease burden.

PHR Use by Authorized Representatives. PHR use by authorized representatives was not a part of this evaluation but will be important for future studies as many beneficiaries have expressed desire for this feature.  Additionally, it will be important to identify the best mechanism for allowing authorized representatives’ access to the PHR. Previous studies have shown that users often desire to have additional control over what kind of information authorized representatives are allowed to view. For instance, users have reported the desire to mask specific information within the PHR. PHR developers should consider what additional features and functions the tool should support to facilitate use by authorized representatives. 

Providers’ Access to the PHR. The successful expansion of PHRs can be dependent upon their uptake and adoption by providers. The impact of PHR use when providers have access to the PHR will be a critical component to measure in PHRs which offer this feature, as this is consistently cited in our findings as a central component to making PHRs useful for beneficiaries. In our evaluation, we found that beneficiaries would be more likely to use PHRs if it was recommended by their providers. However, providers remain reluctant to use PHRs due to issues of reliability of PHR information, liability considerations related to reliability and accuracy, workflow concerns, time constraints and lack of reimbursement.

Usage Data to Track.

The results of future analyses to evaluate PHRs could be vastly improved by the inclusion of certain types of additional data.  The effectiveness of outreach efforts could more easily be measured if, when registering, beneficiaries were asked for a referral source.  This could take the form of a general survey question offering multiple choices for source of referral. Alternatively, specific outreach events could provide numerical codes corresponding to that event, which a beneficiary would then use to register for MyPHRSC.  Customized URLs or referral links could be used in a similar manner. 

Additional demographic data may be useful in conducting multivariate analyses to predict usage of PHRs.  In particular, data on household composition, income, assets, or beneficiary race and ethnicity, for example, could reveal predictive variables that may better explain beneficiary use of PHRs.  Information on utilization of health care services may also be linked to usage of PHRs.  An examination of beneficiary utilization of health care services will be included as part of the evaluation of the Medicare PHR Choice pilot program.

Data on page views that are aggregated daily or perhaps even by session can provide a significantly richer information source than page views aggregated quarterly.  Ideally, a user's navigational path through the system would be captured so that beneficiaries’ activities could be compared to look for differences in usage inside the PHR product. 

Finally, a satisfaction and/or usability survey could provide valuable information on usability or other challenges, and could potentially explain patterns of use and nonuse highlighted in this analysis.

Appendices

Appendix 1. PHR Expert Panel Members

_{Back to Top}

Name	Relevant Experience
William Bernstein, Esq.	Member of Manatt, Phelps & Phillps, LLP’s Board of Directors, Chairman of the Healthcare Division, and  Administrative Partner of the New York office.  Experience includes advising clients in the healthcare industry, including provider organizations, payor organizations, emerging companies and financial institutions.
Michelle Dougherty, RHIA, CHP	Director of Practice Leadership for the American Health Information Management Association (AHIMA). Provides professional expertise to AHIMA members and outside organizations on health information practice issues and develops written products aimed at furthering the art and science of health information management (HIM). Represents AHIMA and the HIM profession at HL7 and on the EHR technical committee.
Ed Fotsch, MD	CEO of Medem, founded in 1999 by six of the nations medical societies and the AMA to deliver the premier secure online physician communications network in the U.S. Led the formation of the eRisk Guidelines.
Archelle Georgiou, MD	Independent consultant and former executive vice president for UnitedHealth's Optum is an expert on PHRs and other health information technology issues from the perspective of insurers.
Brad Hesse, MD	Chief of the National Cancer Institute's (NCI's) Health Communication and Informatics Research Branch (HCIRB). Expertise in PHRs and other consumer-facing research and applications.
Paul Kaplan, MD	Vice President and Chief Medical Officer of Blue Cross Blue Shield of Delaware. Experience in electronic health record implementation.
Steve McPhilliamy	Partner at Insight Product Development, and developer of medical devices and products used by both providers and patients.  Expertise in visual language development, product positioning and managing design programs.
Don Mon, PhD	Vice President of Practice Leadership at the American Health Informatics Association (AHIMA) and Co-Facilitator for HL7 PHR Work Group that produced the PHR-S functional model. Expertise in clinical data standards, PHRs and EHRs, and other health IT issues.
Lori Nichols	Developing prototype PHR for large integrated delivery system
Steve Ross, MD	Assistant Professor at the University of Colorado Health Sciences Center. Expertise in medical informatics.
Rob Tennant	Senior Policy Advisor for Health Informatics with the Government Affairs Department of Medical Group Management Association. Expertise in federal legislative and regulatory electronic data interchange issues, HIPAA, telemedicine, and other health IT issues.

Appendix 2. MyPHRSC Screenshots

Appendix 3. Environmental Scan Key Informants

Name	Organization	Expertise
Gary Marchionini	University of North Carolina	Standards
Jan Oldenburg	Kaiser Permanente	Usability, design, implementation
Rashida Fleming	VA (myHealtheVet)	Implementation
Patti Brennan	Project HealthDesign, UW	Implementation, usability design, standards
Brad Hesse	NCI	Standards, provider perspective
Steve Findlay	Consumers Union	Consumer perspective
Don Mon	AHIMA	Standards
Joyce Dubow	AARP	Needs of the Medicare population
Lori Nichols	Whatcom Health Information Network	Usability design, implementation
Tre McAllister	Dell	Standards, design
Stefanie Fenton	Intuit	Usability design
John Boden	LifeLedger	Standards and implementation
Charles Parisot	Integrating the Health Enterprise	Standards
David Lansky	Markle Foundation	Utility, Implementation and Design

Appendix 4. Contractor Discussion Protocols

Contractor Key Informant: _____________

Contractor Organization: QSSI

The Office of the Assistant Secretary for Planning and Evaluation (ASPE) in the Office of the Secretary at HHS has contracted with NORC at the University of Chicago, a non-profit social science and policy research organization to evaluate CMS’ personal health records (PHR) pilot for fee-for-service Medicare enrollees. 

As a part of this evaluation, we wanted to speak to members of the contractor team that is implementing the pilot project for PHR.  We are interested in learning about your experiences both before and during this project with respect to the PHR’s usability and utility, best practices for design and development and the relevant standards for security and privacy, interoperability and content.

Thank you for taking the time to speak with us about your experiences on this initiative.  We estimate that this interview will last no longer than 45 minutes.

Discussion Guide

1. What is your role on this pilot demonstration project?

2. As the prime contractor what has QSSI learned about the process of coordinating the flow of Medicare claims data into PHRs being targeted to Medicare beneficiaries?

3. What are the largest challenges you faced on this project, and were the challenges expected or unexpected?

4. What role has QSSI played in outreach and dissemination?

5. What do you see as the key challenges that remain on this project?

6. What are the key lessons learned that you have taken away from this project?

7. What do you know now that you wished you knew at the start of the project?

Contractor Key Informant: ___________

Contractor Organization: HealthTrio

The Office of the Assistant Secretary for Planning and Evaluation (ASPE) in the Office of the Secretary at HHS has contracted with NORC at the University of Chicago, a non-profit social science and policy research organization to evaluate CMS’ personal health records (PHR) pilot for fee-for-service Medicare enrollees. 

As a part of this evaluation, we wanted to speak to members of the contractor team that is implementing the pilot project for PHR.  We are interested in learning about your experiences both before and during this project with respect to the PHR’s usability and utility, best practices for design and development and the relevant standards for security and privacy, interoperability and content.

Thank you for taking the time to speak with us about your experiences on this initiative.  We estimate that this interview will last no longer than 45 minutes.

Discussion Guide

I. Background

 

1. What is your role on this pilot demonstration project?

2. For how long has the HealthTrio PHR been available?

II. Project Related Questions

3. How did you gather user requirements and an understanding of what functions to make available to Medicare beneficiaries?

4. What changes were made to the PHR to support the CMS pilot? 

1. Can you describe what, if any, changes were made to the security of the system to meet CMS requirements?

5. Can you describe the process of how HealthTrio receives data from Palmetto?
1. How do you verify the information that you have received has been updated correctly in the PHR?

6. Can you share with us what tools you are using to transform data from claims into information that consumers will be able to understand?

7. Was any initial usability testing done with the Medicare population?  When? How? 

8. Based on the experience with MyPHRSC:
1. Which functions do Medicare beneficiaries like best/least?
2. Are there functions they are not being used? 
3. Are there any features available in the HealthTrio tool but not in this pilot that you believe would have been useful to the users?

9. What have you learned about the unique requirements that FFS users have in terms of:
1. Usability?
2. Utility/functionality?

10. From the information gathered from the technical support line for Medicare beneficiaries:
1. What are some of the most common problems encountered by beneficiaries?
2. Do users have concerns related to the security/privacy of the system?
3. What is your normal turnaround time on resolving tickets?
4. What, if any, requests have you received for additional features and functions?

 

III.      General Questions

1. What usage data is HealthTrio collecting? Can this data be made available to NORC?

2. What specific standards does the HealthTrio PHR support?

1. Are you using the CCR standard for the record summary?
2. What security standards are being used?
3. What, if any, content standards are being used?

3. What are the largest challenges you faced on this project, and were the challenges expected or unexpected?

4. What do you see as the key challenges that remain on this project?

5. What are the key lessons learned that you have taken away from this project?

6. What do you see as the key learning opportunities that remain on this project?

Contractor Key Informant: __________

Contractor Organization: Palmetto

The Office of the Assistant Secretary for Planning and Evaluation (ASPE) in the Office of the Secretary at HHS has contracted with NORC at the University of Chicago, a non-profit social science and policy research organization to evaluate CMS’ personal health records (PHR) pilot for fee-for-service Medicare enrollees. 

As a part of this evaluation, we wanted to speak to members of the contractor team that is implementing the pilot project for PHR.  We are interested in learning about your experiences both before and during this project with respect to the PHR’s usability and utility, best practices for design and development and the relevant standards for security and privacy, interoperability and content.

Thank you for taking the time to speak with us about your experiences on this initiative.  We estimate that this interview will last no longer than 45 minutes.

Discussion Guide

1. What is your role on this pilot demonstration project?
2. Can you walk us through the steps that you go through to populate Medicare Claims data into the HealthTrio PHR?
3. What have you learned about the functional and technical requirements for exporting CMS claims data into a PHR?  How does it differ from exporting CMS claims data to other types of systems?
4. Can you describe what if any transformation to the data was necessary before it could populate the HealthTrio PHR?
5. What have you learned about the security requirements for exporting CMS claims data into a PHR?  How does it differ from exporting CMS claims data to other types of systems?
6. What is the time-lag between receiving claims information and populating the HealthTrio PHR?
7. How does Palmetto verify that the accuracy of the information that is populating in the PHR?
8. Has Palmetto found that any of the data that is populating the PHR may not be readily usable by consumers, as opposed to clinical and administrative professionals?
9. What are the largest challenges you faced on this project, and were the challenges expected or unexpected?
10. What do you see as the key challenges that remain on this project?
11. What are the key lessons learned that you have taken away from this project?
12. What do you see as the key learning opportunities that remain on this project?
    

Appendix 5. myphrsc Users Discussion Guide

Personal Health Record (PHR) Pilot for Medicare FFS Beneficiaries

 

FACILITATOR’S GUIDE 1: BENEFICIARIES AND CAREGIVERS

My Personal Health Record South Carolina (My PHRSC)

 

Background

Purpose: To evaluate the usability and utility of the PHR pilot, and to gain insight into issues pertaining to standards, design, and development principles used for the PHR.

Project Summary: The Office of the Assistant Secretary for Planning and Evaluation (ASPE) has contracted with the National Opinion Research Center (NORC) to conduct a study to identify PHR features that are most valued by end-users, as well as best practices in designing PHRs. 

I.  WELCOME & INTRODUCTION — 5 MINUTES

1. Key Intro Points:
2. Introduce self
3. We work at NORC (NOT CMS) and ASPE sponsors this study
4. Goal of the meeting: Learn about your experiences using MyPHRSC; likes/dislikes; usefulness etc so Medicare can design a PHR best suited to meet your needs
5. Remind them that MyPHRSC is just a pilot but there are many PHRs available that are similar
6. Feel free to express positive or negative opinions
7. All info you share is confidential
8. Not required to share PHI or conditions/diagnoses (and we do not have access to this info)
9. Note-taking without attribution
10. Audio-recording, but not video taping
11. Rules of discussion- talk one at a time; can disagree respectfully; share honest views, etc
12. End at about 2:00pm, can pick up payment at that time
13. Bathroom breaks OK
14. Questions?

You have been invited here today because you signed up with South Carolina Medicare to use MyPHRSC, or someone else gave you access to his or her Personal Health Record.  MyPHRSC is a secure and confidential electronic Personal Health Record.  It lets you get information on all of your Medicare bills over the past 24 months. 

[FACILITATOR] Before we begin, I’d like to get a sense of how familiar everyone is with My Personal Health Record, South Carolina.

1. How many of you have been using MyPHRSC on a regular basis since signing up?
2. How many of you have used MyPHRSC in the last month?

[Based on the answers, facilitator decides if demo is necessary.]

II. OVERVIEW OF MYPHRSC -5 MINUTES

We would like to show you some examples of what you can do with your PHR.  At this time, please get out your packets.  We have used made up information for a made up person so that you can see the kinds of things you might encounter using MyPHRSC.

[Give a brief review of the content of each page in screenshot packet.]

III. FACILITATOR GUIDE:  QUESTIONS 

Group Introductions, 10 minutes

First, I would like us all to introduce ourselves.  Please tell us:

• Your first name
• What is most important to you about your health or healthcare?

Topic 1:  Overall experiences with computers - approx 10 minutes

1.       Overall, how comfortable are you using the internet or going online?

• Can do it on my own and do routinely
• Can do it on my own but do it only occasionally
• Need help from my friends or family
• I have never been “online”

2.       What kinds of things do you typically use the internet to do?

• Banking
• Shopping
• Browsing for information: e.g., news, sports, weather
• Find information about my health or health care

Now, I would like us to discuss potential benefits of using MyPHRSC.  We will talk about whether using MyPHRSC has changed the way you look for and use health information, such as keeping track of your doctor visits or medications. We will also discuss any problems you have experienced using MyPHRSC.   

Topic 2: Perceptions of PHR Utility -15 Minutes

a.    How often do you use MyPHRSC?

[PROMPTS TO STIMULATE DISCUSSION]

• When did you or your caregiver sign up for it?

[For caregivers] When were you given permission to use MyPHRSC?

• How long did it take you to learn how to use MyPHRSC easily?
• Have there been changes over time (i.e., since you first signed up or were granted access) in how often you use it?  That is, once you became familiar with MyPHRSC, did you use it more or less often than you did at first?

b.    Why do you use MyPHRSC?

[PROMPTS TO STIMULATE DISCUSSION]

• To manage my health information
• To view my illnesses and conditions
• To view my health care visits or procedures and immunizations
• To learn more about health related information that is important to me 
• To keep track of my prescriptions and over-the counter medications
• To keep track of my health care appointments
• To share information with my family members, caregiver(s) or health care providers

c.    How does MyPHRSC change the relationship between you and the people who help you with your health or healthcare, such as family members or caregivers?

[PROMPTS TO STIMULATE DISCUSSION]

• Improved communication
• Improved ability to manage the patient’s care
• Any complications?

d.    How does MyPHRSC change the relationship between you or your caregiver and health care provider?

[PROMPTS TO STIMULATE DISCUSSION]

• Improved communication
• Less visits necessary
• Any complications?

e.    How do you feel about sharing your Personal Health Information?

[PROMPTS TO STIMULATE DISCUSSION]

• My information is secure
• My information is not secure
• I don’t know

f.     What changes, if any, would you make to MyPHRSC, and why?

 [PROMPTS TO STIMULATE DISCUSSION]

• Laboratory results
• Advance directives
• Give my provider access to my PHR
• Access to a log that indicates who accessed my record

g.    Where do you use MyPHRSC?

h.    Is there something else you think needs to be included in your personal health record?

Next, we are going to talk about WHY you use the PHR, but we are also going to talk about HOW you use it. 

Topic 3: Perceptions of PHR Usability-15 minutes

a.    Do you know how to find the MyPHRSC website (the URL) on the internet? Why do you find it (easy/hard)?

• Forget the website link
• Have it bookmarked only on one computer
• Do you know how to log in?
• Do you need help from friends or family to use MyPHRSC?

b.    How easy or difficult is it for you to use MyPHRSC? 

           [PROMPTS TO STIMULATE DISCUSSION]

• Are the instructions clear?
• What part of MyPHRSC do you find the easiest to use?  Why?
• What else do you think MyPHRSC should be able to do for you?
• How easy or hard is it to read and understand the information in MyPHRSC?
• Right font size so it is easy to see
• Plain English
• Easy to understand
• Colors easy to read
• How may clicks does it take to get to the specific functions in MyPHRSC?

c.    Are you concerned about privacy and security issues related to using MyPHRSC?  If so, what risks are most troubling to you? 

• General security breaches
• Insurers and employers learning about health problem
• Feel comfortable with the security of electronic communication with their provider
• Any other concerns

d.    Did you see anything that surprised you in MyPHRSC?

• Did you know who to call if you had problems/questions/or disagreed with data in the system?

Part of this study’s purpose is to learn more about whether using a Personal Health Record affects the health of Medicare beneficiaries.  We would like to hear about ways that you are using MyPHRSC to do things like prepare for doctor visits, manage medications, or get health information.

Topic 4: Perceived Impact of PHR Use on Health Status-15 minutes

a.    Do you ever use MyPHRSC to get ready for a visit to your doctor?

• Does it help you to ask questions?  If so, how?
• Does it help you to remember what medications the doctor has given you?
• Does it help you to talk with the doctor about why you are seeing the doctor and what he is going to do?

b.    Can you tell us how you use MyPHRSC to keep track of your medications? 

• Remembering how and when medications should be taken
• Checking/ confirming how much medication to take
• Checking for interactions with my medications
• Sharing information in “My Medications” with my doctors or caregivers (allergies, interactions, or new over the counter medications)
• Checking refill information
• Keeping track of my vitamins, herbs, and over the counter medications

c.    Can you share with us some examples of times when your information in MyPHRSC has been valuable for you?

d.    Do you think the use of MyPHRSC has had a positive impact on your health, how you use your Medicare, your communication with doctors, or your own care at home?

[Do not offer prompts until beneficiaries have given their opinions.]

• Keep better track of my medications
• Keep my contact information up to date, e.g. address, phone number
• Helps me talk with my doctors
• Keep better track of my visits
• Helps me keep track of how much I pay for my healthcare
• Gives me facts about my health
• Fewer repeat tests
• Fewer unnecessary hospital/ER visits
• Negative impact?

e.    Have any of you given permission to caregivers, friends, relatives, or health care providers to access your records on MyPHRSC? 

(MODERATOR should navigate to “My Permissions” page.] 

If so, to whom did you let look at your records and why?

f. (MODERATOR should pose this question to caregivers)  To what extent, if any, does MyPHRSC help you manage care across providers for the person you take care of?  What else could MyPHRSC do to help you arrange care for the person you care for?

Part of this study’s purpose is to learn more about whether using a Personal Health Record affects the health of Medicare beneficiaries.  We’d like to hear about ways that you are using MyPHRSC to do things such as prepare for doctor visits, manage medications, or get health information.

Topic 5: Consumer Satisfaction with PHRs-15 minutes

a.    Overall how satisfied are you with MyPHRSC?

b.    Do you feel that using MyPHRSC will improve your health or the health of the person you care for in any way?

c.    What changes would you make to the PHR to make it better or more useful to you?

d.    Would you be willing to pay to use a personal health record such as MyPHRSC?

• How much would you be willing to pay per month?

e.    How current and accurate is the information in your PHR?

• Are you able to find the information that you need?
• Are you able to find information that is useful to you?

IV. Wrap Up — 10 min

Pearls of wisdom

Thank you, this has been a very interesting discussion, etc

We may be contacting you to follow up in a couple months for user observation studies (one on one) — would that be okay with everyone?

We are also contacting doctors in the Columbia area to ask their opinions of personal health records.  If you would like to provide us with the name of your local doctor, Ali is keeping a list and she would be happy to take your doctor’s name.

• [We will not ask doctors any information about you, nor will we share any of the info you have shared today with the docs.  They will not know that we have spoken with you.] 

Synthesis of findings/ development of project report

Appendix 6. discussion Guide for Nonusers

Personal Health Record (PHR) Pilot for
Medicare FFS Beneficiaries (Non-users)

 

TELEPHONE GUIDE: BENEFICIARY MYPHRSC NONUSERS

Background

Purpose: To evaluate the usability and utility of Personal Health Records, specifically My Personal Health Record South Carolina (MyPHRSC) and to gain insight into issues pertaining to use of PHRs by Medicare beneficiaries.

Project Summary: The Office of the Assistant Secretary for Planning and Evaluation (ASPE) has contracted with the National Opinion Research Center (NORC) to conduct a study to identify PHR features that are most valued by end-users, as well as best practices in designing PHRs. 

I.  Welcome & Introduction

Thank you very much for taking the time to speak with me.  My name is [INTERVIEWER NAME], and I work for the National Opinion Research Center (NORC), a national, not-for-profit research organization affiliated with the University of Chicago.  The Office of the Assistant Secretary for Planning and Evaluation (ASPE), Department of Health and Human Services (HHS), has requested that we gather your opinions on personal health records.  We do not work for CMS or for ASPE and are an independent non-profit organization

We are contacting you because you have Medicare health coverage in South Carolina and signed up through Medicare to use My Personal Health Record South Carolina—also known as MyPHRSC.  It is a secure and confidential electronic Personal Health Record.  It lets you get information on all of you Medicare bills over the past 24 months.  We are talking to several people who signed up about their experiences.  We recognize that some people are logging in weekly and some have only logged in once.

Before we start, I would like to take a second to assure you:

There is no right or wrong answer to the questions I will ask.  We are interested in hearing your thoughts, whether they are positive or negative. 

I do not work for South Carolina Medicare or the company that created MyPHRSC, so you will not upset me if you express negative opinions.

We will use your comments, but no names or identifying information, in our report.  All info you share is confidential

Remember, your participation is voluntary, and you may stop the discussions at any time. 

Our conversation today should take about 30 minutes; is this still acceptable for you?

Recently Medicare started a program in South Carolina to give people free access to a safe and secure electronic personal health record called My Personal Health Record, South Carolina (MyPHRSC).  A Personal Health Record can be many things.  Generally, a personal health record is considered a collection of information about your health or health care services (e.g., provider visits, hospital visits, medical conditions, and your own notes). Personal Health Records may be in paper format (e.g., records you keep at home) or in an electronic format (e.g., records stored on the computer or on the internet).  It is designed to help you manage and understand your health information and share it with other people if you would like.

1.    What healthcare issues are most important for you? 

• Reducing healthcare costs
• Managing my chronic conditions
• Access to help or care when I need it
• Maintaining my healthcare regimen more effectively (taking medications on time, etc)
• Understanding my conditions
• Any other issues?

2.    What kinds of things do you typically do to manage your own healthcare?

• Are you involved in services that promote patient education, provide you with email updates, allow you to manage your appointments, or encourage your participation in preventive services--including mammogram screenings?
• Do you keep a diary for pain, other symptoms, and side effects?  Do you use longitudinal health tracking tools such as charts, graphs for BP, BS? Do you monitor your prescriptions for potential drug interactions, what about your prescription refills, do you keep track of those?  Do you track your claims information?
• Do you generally read your explanation of benefits? )
• Would you consider using a PHR for any of these things?  Why or why not?

3.    Overall, how comfortable are you using the internet or going online?

• Can do it on my own and do routinely
• Can do it on my own but do it only occasionally
• Need help from my friends or family
• I have never been “online”

4.    What kinds of things do you typically use the internet to do?

• Banking
• Shopping
• Browsing for information: e.g., news, sports, weather
• Find information about my health or health care

5.    How did you find out about MyPHRSC?

[PROMPTS TO STIMULATE DISCUSSION]

• Workshops
• Health fairs
• Information received in the mail
• The news
• Other venues?

6.    Did you or someone else sign you up for the program?

• Yes, I signed myself up for it
• Yes, someone else signed me up for it

7.    When you decided to sign up for MyPHRSC, what did you hope to get out of using it?

• To manage my health information
• To view my illnesses and conditions
• To view my health care visits or procedures and immunizations
• To learn more about health related information that is important to me 
• To keep track of my prescriptions and over-the counter medications
• To keep track of my health care appointments
• To share information with my family members, caregiver(s) or health care providers

8.    Since signing up for MyPHRSC you have not accessed the PHR. What are your reasons for not accessing the PHR?

9.    Would you be interested in using a personal health record like MyPHRSC on a regular basis?  Why or why not?

• Do not need to use it
• Do not have regular computer access or know how to use one
• Yes, because it could help me to manage my medications, etc
• It is too hard to use
• I will forget

10. If you could design a personal health record and include the features and services you want to track your health, what kinds of things would you include?  (services such as patient education, emailing, appointments, preventive services mammogram screenings, patient diaries for pain, symptoms, side effects, longitudinal health tracking tools such as charts, graphs for BP, BS, drug interactions checking, claims information such as an explanation of benefits, or Rx refills; features such as…get examples from HealthTrio). Of your suggestions, which do you think would be most useful for Medicare beneficiaries?

Thank you very much for your time today.

Appendix 7: USER OBSERVATIONS Discussion Guide

Personal Health Record (PHR) Pilot for Medicare FFS Beneficiaries

LABORATORY OBSERVATIONS OF MYPHRSC USERS

1.  Introduction — 5 minutes

We want to start by thanking you for coming today and agreeing to share your thoughts.

We are here today because we work for the National Opinion Research Center (NORC) at the University of Chicago and are conducting a user observation study about personal health records sponsored by the Assistant Secretary for Planning and Evaluation.  We have spoken with you before (by discussion or telephone) and are trying to gain insights into how the personal health record (PHR) fit into your home life, relationships with your health providers, and overall healthcare delivery process.

Today, we will be asking you questions in the form of a causal interview.  This entire observation will last about 2 ½ hours, and you will be provided with compensation in the form of a $75 check. If you returned the health booklet, you will receive an additional $75 check, for a total of $150.

It is important to understand that if you choose to participate, all the information you provide will be held in confidence.  Our conversation and observation will remain absolutely confidential, meaning we will not share what you specifically have to say with your health plan, Medicare, or anyone who is not on the project staff.

We would like to record this session so we can accurately capture our conversation. However, if you would prefer, we can take written notes. Also, if you agree, you may ask to stop the recording at any time, and we will turn off the machine.  If you decide to stop taping, we will ask your consent to retain the portion already taped. Is it OK if we audio record this session?

Your participation is entirely voluntary.  You may leave at any time without penalty, and you may choose not to answer any question or participate at any time. Should you choose not to participate this will not affect your Medicare benefits in any way.

This is a quick summary of what we want to do today:

• We will spend some time talking with you about your personal background and your medical status. We also want to talk to you about what tools you use to manage your information—such as paper and computer—and what technologies you use for personal and healthcare use.
• We will walk through the features on MyPHRSC and discuss what you find useful and what you do not.

Do you have any questions for us before we get started?

2.    Background - medical, typical day, role of caregivers — 10 minutes

• What is your typical day like?
• Daily routine?  Activities that you engage in.
• Who lives with you?

Medical related tasks that are a part of this routine (exercise, treatment, medications, appointments with providers, etc)

• What is your current medical overview?
• Are there family and/or friends involved in helping with your care?
  [[reference journal question 7]]
• Do they have legal authorization to decide about your care (healthcare power of attorney)
• What kinds of things do they do to help? 

(Get specific, recent examples of things that the caregivers have done.)

Ask the above general question, and then probe on the below individual items based on their response.

• Make medical appointments for you?
• Track your medical-related finances
• Transport you on medical-related trips (e.g. pharmacy and doctor)
• Attend doctor’s office visits?  Help select providers?
• Help or make decisions regarding
• Help with post-visit compliance?
• Do independent research into healthy lifestyles, preventative care and/or treatment options?

How have you let other people who help you see your health information?

• Why did you decide to do this?

 

3.    Other Health Technology / Comfort with Technology (in general) — 15 minutes

• What is your favorite piece of technology? What is your least favorite?
• Probe: Tell me about a situation where technology has failed you / made you uncomfortable 
• How do they self describe themselves in terms of technology proficiency and comfort (computer or otherwise)?

Do you use the Internet?

• Where do you access the Internet?
• Type of connectivity?

How do you use the Internet?

• What sites do you use?  What do you use the web for? (E.g., Shopping, banking, searching for health information?  Do you use the internet as a general source of information?)
• Do you go online to find information about your health or healthcare?

Are there other technologies in your household (e.g. DVD, Tivo, Nintendo Wii, cell phone, PDA)

• For what purposes do you use them (medical and non-medical)? 
• When do you use them?
• Why did you decide to use these technologies?  (Get at sources of motivation)

Other health-related technologies (glucometer, other home treatment or diagnostic devices)

• How did you learn to use these technologies?
• Where is the information from these devices recorded?
• Is data from the devices ever reported back to the providers?  (automated, logged by beneficiary on  chart, etc.)
• Why did you decide to use these technologies?  (Get at sources of motivation)

4.    Health Information Management — 40 minutes

• The last time you had a health problem or concern, how did you find out more information about that problem or concern?
• Let’s go over health booklet to discuss how you organize your health information. (Focus on WHY… why do you keep your information here? Do you have a system? Is it categorized, etc.?)
• Please describe where you keep your medical records (including medical bills, lab results, prescriptions, etc.). For example, you may store them in a file cabinet in your bedroom, or in a drawer in your kitchen.
• Please describe where you keep your general book-keeping records (e.g. general bills, banking information).
• Please describe where you keep contact information for your providers (e.g. phone number for pharmacy, address of doctor’s office).
• Please describe where you keep medical information that you may need to access in the event of an emergency.
• Please describe where you keep general information regarding your health or medical conditions. For example, if you have diabetes, where do you store information related to the different types of treatment available for diabetic patients?
• Please describe where you keep medicines that may have been prescribed by your providers and where do you store over-the-counter medications (e.g. in the bathroom cabinet, on the bedside table).        
• Do you keep any health information on your person (e.g. in a purse or wallet)? If so, what information do you keep and how is it stored?
• Where do you keep any general health-related information that you use (e.g. health journals, newsletters, books in a family library, magazines)
• Where in your home is your computer? If you use a computer outside of your home, please describe where that is located.
• Please explain anything else that explains how you organize your health information.

How accessible is information when it is stored in different locations? When do you use different information?

Is information you use often stored in easier to access locations? What locations are those? What information is that?

What information do you use rarely? Where do you store it?

• (Based on the participants’ response, the researcher can organize the information, following the discussion, according to the four types of personal information management techniques noted in the JAMIA article: Just-in-time, Just-at-hand, Just-in-case, Just Because.)

Tell us how using MyPHRSC changed the way you keep track of your records.

Tell us about the challenges you face in organizing your health information.

[[Reference Journal, “Challenges” questions]]

1. Please write about a time when you could not find your health information.
2. Have you ever lost important information? Have you ever felt frustrated when trying to organize your health information?
3. What are the major challenges for organizing your health information? Tell us about any successes you have found in organizing your health information. [[Reference Journal, “Successes” questions]]
4. Please write about a time when you had all the health information you needed. 
5. What do you find particularly useful for organizing your health information? Are there specific locations or tools that help you?
6. Is some information easier to organize than other information?

5.       Current Use of PHR - when, why, who, where; other users; role PHR plays in health management; perceived value; impact on relationship with caregivers and providers — 25 minutes
Have you used other electronic personal health records like MyPHRSC?

• How often do you use MyPHRSC? Have you used it this month?
• What locations and computers have you used to access it?

Think about the last time you that you had a health problem or concern that you thought MyPHRSC might be helpful with.  Did you decide to use MyPHRSC to help with this? Why or why not?  How satisfied were you with the help you got from using MyPHRSC [or from whatever else they did]?  If you did not use MyPHRSC to help with this health concern, are there are other health or healthcare issues you think MyPHRSC could be helpful with? 

• [If they report being satisfied using MyPHRSC above]: Tell me about how this was helpful for you.  Do you think using MyPHRSC had a positive impact on your health or healthcare? 
• What was appealing that drove you to switch to MyPHRSC?  When did the issue occur?  Is this issue still relevant to you now?

In thinking about your experiences using MyPHRSC, what suggestions do you have that would improve the PHR? What would make this more useful for you?

• How has the PHR impacted your relationship with the people who assist you with your healthcare? (e.g. quality of communication, level of health educational awareness, etc.)
• How has it changed your interactions, for better or for worse?
• How has the PHR impacted your relationship with your provider?
  
• Have you shared information from it with a provider?
• How did you share this information?
• How would you have previously shared this information?
• For better or for worse?
• Has the PHR prepared you to better communicate with your doctor/provider?
• Has your PHR helped improve your understanding of your medical conditions and medical status?
• Emotional Affinity: This activity provides a framework which allows participants to talk about some of the emotions they experience when using MyPHRSC. We will use some stock images affixed on two 11 x 17 pages, to get reactions from users on which images most closely related to how they feel about MyPHRSC. 
• Person speaking to a pharmacist
• Police officer pulling over a motorist
• Family eating dinner
• Book club or other type of discussion
• Parent talking to child
• Person ordering fast food
• Sitting in traffic
• Person talking to a retailer who offers advice (butcher, hardware store clerk, etc.)
• Visiting a library

Use these images to tell us about some of the feelings you experience when using MyPHRSC

• Which photos can you relate to MyPHRSC?
• Which photos clearly do not relate?

Use these images to tell me how you would like to feel when using MyPHRSC (future emotional desires)

• Which photos can you relate to MyPHRSC?
• Which photos clearly do not relate?
• Would you like to keep using the PHR?  Is this a valuable solution for you, and why?
• Would you pay a monthly fee to keep using it?  If so, how much?
• Is there anything that could be changed about the PHR that would make you willing to pay (or pay more) for the PHR? For example, would you be willing to pay for a “concierge” or “coach” service to help you gather your personal health information?
• Do you know anyone else using a PHR?  How are they using it?  How does it compare to your experience with MyPHRSC?

6.       Key Features, Key Challenges - likes / dislikes, cognitive human factors issues, design, discoverability, navigation, etc — 45 mins

Now we’d like to walk through the features of the PHR and observe you use it. Please talk aloud to explain what’s going through your mind. We are going to start with a couple scenarios and see if you find these features useful.

Questions for moderator to consider throughout the scenarios:

• Users can also be asked/observed on the ease of use of each feature.
• Go from easier to harder functions. Look out for user frustration
• How does the user approach the feature?  What do they talk about when they use the feature?
• Have the users discuss the value of different features and to prioritize them.
• How would they tie this feature into their current personal information management practices (offline or online)? Other medical management devices (e.g. glucometer)?

Health Record Summary

• Ask user to access their health record summary
• Illnesses and Conditions
• Ask user to access this page
• Ask user to add a condition
  
• Visit Summary
• Ask user to access visit summary
• Medications:
• Please use the formulary
• Ask user to enter medications into the system.
• Wallet Card:
  • Please make a wallet card and show how you would print it out.
• Provider Search
  • Find a new provider.
• Health Calendar
  • Pretend like you are about to go to an appointment with the new doctor you found. First, check what time your appointment is on the Health Calendar.
• Claims
  • Now, you’ve returned from your appointment and want to check the Medicare claims from that visit, and the record of the visit summary.
  • What value do you get from the CMS claims data? Do you currently use myCMS.gov for this same claims data, and how would you contrast the two tools for accessing some of the same data? 
• Authorized Representative and Permissions:
  • Add an authorized representative and tinker with their ability to see your information.
  • How could this feature be more useful?
• Healthwise and Trusted Websites
  • Look up information about an ailment. Which resources do you find more useful?
• Please look at these additional pages:
  • Health Assessment
  • Frequently Asked Questions
  • User Preferences

7.    Changes / Improvements / Unmet Needs / Additions / Concerns (privacy, security, reliability, etc) — 10 mins

• Do you have any concerns about the availability of the MyPHRSC website, or have you had any problems accessing it?
• Do you have any concerns about using an electronic personal health record?
• How do you feel about the privacy and security of your information in MyPHRSC?
• Would you be more or less likely to use a PHR sponsored by the following, and is this due to privacy/security concerns?
• CMS
• Another government agency (state or federal)
• Your provider
• Other source

8.    Describe typical health management tasks (in home) - discuss how these activities should link to PHR (e.g. taking medications, vitamins, injections, recording blood pressure in a journal). — 5 mins

• Is there any current connection between these home health management tasks and the PHR?
• Is there any way that they are connected?  Should be connected?
• How has this or could this make tasks better

Appendix 8: Discussion Guide FOR PROVIDERS

Personal Health Record (PHR) Pilot for Medicare FFS Beneficiaries

 

TELEPHONE GUIDE: PROVIDERS

Meeting Discussion Guide —
Providers of Medicare Beneficiaries — PHR Users

OVERVIEW of MyPHRSC -5 minutes

[ALI]  We would like to show you some examples of what Medicare’s PHR, MyPHRSC, offers for beneficiaries.  At this time, please turn your attention to the WebEx presentation.  We have used made up information for a made up person so that you can see the kinds of things you might encounter using MyPHRSC.

[Give a brief review of the content of each page in screenshot packet.]

• Group Introductions, 15 minutes

Before we begin, I’d like us all to introduce ourselves. Please tell us:

• Your first name
• Your medical specialty and a little about your practice

Conduct Discussion, 60 minutes

…Let’s start by talking about your overall experience
with personal health records approx 15 minutes

1. In your opinion, what is a personal health record?
   • Who is the audience, what is the format, where is the information stored, what kinds of data does it contain, etc
   • What is the purpose?
   • How would you use a PHR?
2. Have any of your patients indicated to you that they have used a PHR? [PROMPTS TO STIMULATE DISCUSSION] [If any of the providers have patients using a PHR, ask them to describe their experiences.]
• Roughly what percentage of your patients are using PHRs?  Do you know how long they have been using them?
• If your patients have a personal health record, how has interaction with your patients been different because they have it?
• What functionalities of personal health records do patients indicate they like or dislike the most?
• What are the ages of your patients using PHRs and what types of patients do you see using and/or benefiting most e.g. seniors, those with co-morbidities, the chronically ill.
3. Have any of your patients shared with you information relevant to their healthcare that they obtained using the internet? How does this affect you or your practice? What types of information do patients share? 
   (e.g.: One patient printed articles off of PubMed which he believed served as a second opinion that his doctor’s diagnosis was wrong; another printed information about a medication they wanted to try.)
4. Have patients shared their own personal written records of historical health events with you at visits? How does this affect you or your practice? What kinds of records do patients share, and how accurate or useful is the information?
• MyPHRSC offers a wallet card that summarizes conditions, recent visits, emergency contact information and medications.  How would it affect you or your practice if your patients carried this information to visits and shared this with you?
5. Have you ever communicated with patients through email? Some PHRs allow patients to directly contact providers by secure email.  How do you think it would affect you or your practice if this feature was available to your patients?
6. Some practices now use EMRs which give patients a view into the information through their PHRs.  For those who have EMRs in practice, how would it change things if this information was shared with patients?
   
• How much clinical information do you think should be included in the PHR? Are there things in particular you would not want a patient using a PHR to have access to? …Now we’d like to get your perception of personal health records approx 30 minutes
7. If your patient has a PHR but your practice does not have an EMR/HER, would the PHR be useful to you?  What do you think are the most common reasons for a physician to integrate PHRs into their practice and treatment of a patient?
   • increase quality of service provided to patients
   • faster/better communication with patients
   • faster/better reporting of diagnostic tests (e.g., radiology or labs)
   • reduces physician or staff time, expense
   • reduces repeat and unnecessary tests
   • reduces medical errors
   • patient education
   [Depending on responses probe further on specific issues e.g., chronic disease management, managing medications, reducing physician/staff labor costs, facilitates medical research]
8. Personal health records have the ability to provide detailed information about diagnosis, medications, tests and other medical background.  What information in a PHR would you find most useful?
   • Are there times when you think there is too much information available to patients? 
   • What type of information should be part of a PHR?  What type of information should NOT be part of a PHR?
9. What features or functions of a PHR would be least useful for you?
10. Under what circumstances would you recommend a patient use a personal health record?
11. What are the biggest barriers to physician integrating with PHRs?
    • Lack of interoperability with EMR/EHR systems
    • Impact to your workflow or cultural change in the practice
    • Reimbursement or compensation
    • Concerns of accuracy of patient-entered information
    • Concerns of patients entering irrelevant or lengthy descriptions
    • Concerns of patient overuse or abuse of secure email privileges
    • Lack of standards
    • Lack of laws and rights dealing with privacy and confidentiality
    • Legal liability concerns
    • Reliance on propriety system
    • Lack of business case
12. Are you concerned about privacy and security issues related to using personal health records?  If so, what risks are most troubling to you?  [PROMPTS TO STIMULATE DISCUSSION]
    • General security breaches
    • Insurers and employers learning about health problem
    • Legal liability
13. If you were to design a PHR, what would you include to make this most valuable to you and your practice? What do you think would be most valuable to providers in general? …Now let’s discuss some of the potential impacts of personal health records approx. 15 minutes
14. Do you think patient PHRs might change how you practice medicine?  How? 
    • Will it change the workflow process in your office?
    • Will it influence your communication/relationship with the patient?
    
    
15. What do you see as the benefits of personal health records for patients? How do PHRs change the provider relationship with patients? How do you feel about it?
    • Better relationship with provider
    • Better self-management of care
    • Improved overall health
    • Keep better track of medications
16. What kinds of PHR features or functionalities could make your interaction with patients more effective? Why?
17. Which PHR functionalities have the potential to significantly impact the health outcomes of patients if used? Why?
18. Would you consider using personal health records in your office in the future?  Why or why not?
• What kinds of things would you use the PHR for?
19. For what reasons would you consider integrating personal health records into your practice?  What incentives would be appropriate for providers from your perspective?
    • Underwriting the cost of the technology
    • Compensation for the extended time in clinical session

Thank you for your participation in today’s discussion.

Appendix 9. Beneficiary Pre-Screener AND RESULTS

First Name_________________________

To help us plan for our meeting on [X date, X time], we’d like to learn about what MyPHRSC means to you.  There is no right or wrong answer to the questions below; everyone’s experience is different.  Please write a brief answer to each question. Use the enclosed, self-addressed envelope return your completed form to XXXXXXX at NORC by X date.

 

1. When did you or the person you care for first hear about My Personal Health Record South Carolina (MyPHRSC)?  Who told you about it? People first heard about it between 1 and 6 months ago. They learned about it from a variety of sources: spouse, newspaper, mailing, internet, or events promoting MyPHRSC.  
2. When did you or the person you care for sign up for MyPHRSC, and why?  People signed up between 1 week ago and 4 months ago. Most people didn’t answer why; those who did said, “to get information.”  
3. How did you expect to use MyPHRSC?  Many people didn’t know what to expect and simply wanted to check it out. Some expected to use it to “keep track of health records,” seeing the advantage of their records being “in one place” or “more up to date.” One beneficiary just wanted to make a wallet card.
4. Has your actual use of MyPHRSC been what you expected?  How has it been different? Many people have not used the program much (a couple haven’t used it at all). One beneficiary critiqued it as too technical and another found it very thorough.
   
   
5. If a friend asked, “What is a Personal Health Record?” How would you describe it? Many people defined it as “keeping health records.” Some felt unable to answer this question, though others praised it as “helpful,” “valuable,” and “would recommend to friends.”

Appendix 10. PROVIDER PRE-SCREENER

1. Do you currently practice medicine in South Carolina?

   Yes _____

   No _____
   

2. What percentage of your patients are currently fee-for-service or original Medicare beneficiaries?   

   25% or more _____

   Less than 25% _____

3. Do you have access to the internet through a home, work, or some other computer?

   Yes _____

   No _____

4. Are you aware of the personal health record offered to Fee-for-Service Medicare beneficiaries in South Carolina called My Personal Health Record, South Carolina or MyPHRSC?
   

   Yes _____

   No _____

5. Have any of your Medicare patients told you they use a personal health record?

   Yes _____

   No _____

6. Does your practice use Electronic Medical Records?

   Yes _____

   No _____

Appendix 11. PROVIDER PRE-SCREENER RESULTS

Specialty	% Medicare Patient	% Fee-For-Service/Medicare	Internet	Aware of MyPHRSC	Patients Use MyPHRSC	Practice uses EMR
Family Practice	45	60	Yes	No	No	Yes
PCP	30	55	Yes	No	No	No
Family Practice	30	25	Yes	No	No	No
Internal Medicine	35	35	Yes	Yes	No	Yes
Optometry	40	50	Yes	No	No	No
Family Practice	40	100	Yes	Yes	No	Yes
Internal Medicine	70	70	Yes	No	No	No
Chiropractic	20	45	Yes	No	No	No
Family Practice	75	75	Yes	Yes	No	Yes

Footnotes

• http://medical-dictionary.thefreedictionary.com/SNOMED).
• http://www.cgu.edu/include/lansky.pdf.
• http://74.125.47.132/search?q=cache:jhSwBnfbLRMJ:www.nahit.org/images/pdfs/HITTermsFinalReport_051508.pdf+nahit+phr+definition&cd=1&hl=en&ct=clnk&gl=us&client=firefox-a#19. Retrieved July 13, 2009.
• http://www.connectingforhealth.org/resources/final_phwg_report1.pdf
• http://www.cgu.edu/include/lansky.pdf.
• http://www.aarp.org/ppi
• http://www.cms.hhs.gov/PerHealthRecords/Downloads/SummaryofPersonalHealthRecord.pdf.
• http://www.cms.hhs.gov/apps/media/press/release.asp?Counter=3401.
• https://beta.myphrsc.com/login.cfm
• http://www.healthtrio.com/
• http://www.statehealthfacts.org/comparemaptable.jsp?ind=290&cat=6
• http://www.markle.org/resources/press_center/press_releases/2005/press_…;
• http://www.phrconference.org/conf_resources/presentations/dec6/phrs_101….
• http://www.connectingforhealth.org/resources/final_phwg_report1.pdf
• http://www.phrconference.org/conf_resources/presentations/dec6/phrs_101….
• http://www.connectingforhealth.org/resources/final_phwg_report1.pdf
• http://xnet.kp.org/newscenter/pressreleases/nat/2009/071409myhealthmana….
• http://www.ncvhs.dhhs.gov/090609p3.pdf
• http://www.statehealthfacts.org/profileind.jsp?cat=6&sub=74&rgn=42.
• http://www.statehealthfacts.org/profileind.jsp?cat=6&sub=74&rgn=42.
• http://www.qualityforum.org/projects/prioritization.aspx
• http://www.pewinternet.org/Reports/2004/Older-Americans-and-the-Interne…
• http://www.statehealthfacts.org/profileind.jsp?rgn=1&cat=6&ind=296.
• http://www.statehealthfacts.org/profileind.jsp?cat=6&sub=74&rgn=42.
• http://www.connectingforhealth.org/resources/final_phwg_report1.pdf
• http://www.kpihp.org/publications/docs/phr_summary.pdf
• http://online.wsj.com/article/SB119565244262500549.html
• http://www.boston.com/news/nation/washington/articles/2009/04/18/beth_i…
• http://www.boston.com/news/nation/washington/articles/2009/04/13/electr…