Evaluation of the Personal Health Record Pilot for Medicare Fee-For Service Enrollees from South Carolina. Usability and Utility of MyPHRSC Features.


The use of MyPHRSC varied, though most beneficiaries had only logged into the program a handful of times prior to the laboratory session. During the user observation studies, participants were watched as they navigated through different features of the tool and described what they found both useful and challenging about it. Attention was also paid to other specific observations made by the participants regarding the usability (the convenience, effectiveness and ease of use) and the utility (the level of usefulness) of the PHR features.

Below, we outline the successes and challenges of using different features of the tool, as described by the beneficiaries.

Health Record Summary.  Most beneficiaries seemed impressed by all the information while others were overwhelmed by the large list of their illnesses and conditions. One beneficiary commented that the summary was, “much, much more thorough than I could ever imagine,” while another said, “Now you’d look at that and you think I'm pretty sick.” Overall, participants seemed glad that this feature provided a list with greater detail about their health—including dates—than the hardcopy Explanation of Benefits report received from CMS in the mail.

The major dissatisfaction with the Health Record Summary came from beneficiaries not recognizing everything listed on the page. For instance, beneficiaries often did not understand all of the medical terms that were used in the “Problems” list. Some participants also noted that they saw conditions that they were unaware they had been diagnosed with. Others saw health providers’ names they did not recognize or facilities’ addresses that were inaccurate. The beneficiaries suggested that the providers whose names they did not know may have completed lab work or other behind-the-scenes tasks, and that the addresses may be for alternative clinics their providers also visit. One participant saw that both a provider and his spouse were listed twice under the “People” section. This participant noted that the Health Record Summary was not editable, nor was there the ability to edit provider information.

Some information on the page seemed superfluous. For instance, the “Problems” list included a column of the SNOMED and ICD-9 codes, which the participants did not understand. The “Status” column was also unclear and in some cases inaccurate, and often the beneficiary no longer had a certain condition (e.g. cough) that was marked as “Active.” In addition, some beneficiaries were missing information. For instance, one had updated an e-mail address on the User Preferences page, but did not see it on the Health Record Summary page as expected.

Illnesses and Conditions.  Beneficiaries had comments about the Illnesses and Conditions page that were similar to those they had with the “Problems” list from the Health Record Summary. Although the page contained a great deal of useful information, including dates and diagnoses, there were some words that were unclear; for example, a beneficiary reported not understanding what the word “hepatitis” meant. Additionally, some beneficiaries had difficulties using the “add” and “edit” features on this page. When a participant tried to search for a condition to add (e.g., claustrophobia), the error message “No Records Found” appeared. Another beneficiary was disappointed by the lack of detail about a condition within the “Illness/Condition Detail” section, which provides no additional information other than that data from the Illness and Conditions list with participants only able to add “notes”. The participant was hoping to find more information about the listed condition on this page, or possibly see details taken from a provider’s notes regarding the diagnosis.

Medications.  The MyPHRSC pilot program did not have access to Medicare Part D data, so medications were not automatically entered into the PHR. As a result, the Medications page proved to be the most challenging. Some of the beneficiaries were unable to enter their medication information without our guidance. Although not included as part of the original ASPE evaluation, CMS recently downloaded medication information from its Department of Defense partners (TRICARE for Life) in South Carolina. One participant who had both CMS and TRICARE coverage was very pleased to see the medication information listed in the MyPHRSC tool. The TRICARE prescriptions were entered each time medications were filled, which was every three months for this beneficiary.

In order to enter medication information into the PHR, beneficiaries first had to find their medications using the search function on the Medications page.  Searching for a medication involved one of two different paths: 1) identifying the category or subcategory that a medication belonged to; or 2) conducting a top level search. This process was confusing for most of the beneficiaries. They often initially attempted to find their medications using the first path, the “category” search, but were unsure about which category or subcategory their medication belonged. Ultimately, none were able to locate their prescription names using this functionality. Following this attempt, beneficiaries moved to conducting a top level search by typing their medication into the search function at the top of the page. One participant was unsure if entering the generic or the brand name drug was appropriate. The selection of “route,” which offers a drop-down menu of forty-four different routes in the way medication is taken was unclear to some. Many of the terms included in this drop-down menu, such as percutaneous, intraperitoneal, buccal and intrapleural, were confusing.

Once they had located their medication, participants often were unsure how to add it to their medication list. One beneficiary presumed that clicking on the medication would make it automatically appear on the medication list. However, another step was required. After beneficiaries searched and selected their medication from the list, one or more medications appeared in blue under the originally selected drug name. The beneficiaries then had to select the correct medication with the appropriate dosage from that list, which brought them to the “Add Medication” page.

"[The medication page] is not as user-friendly for somebody who is in their late 60s, early 70s."


The information on the “Add Medication” page was, in the words of one beneficiary, “cumbersome to enter in manually.” First, the page asks beneficiaries to enter the name of the Prescribing Clinician. Attempts to search for clinicians were often unsuccessful. One participant tried to “drag and drop” a provider’s name from the list on the left panel of MyPHRSC. Another searched for the provider by typing in the name using a “Dr. First Last” format, and the provider’s name did not appear. Within the “Provider Search” page, the abbreviation “PCP” was unfamiliar to many beneficiaries. Beneficiaries then entered the information in the dosage, frequency, and duration fields. Most preferred not to complete the “start date,” as it was hard to recollect the actual start date of a medication, particularly in the context of a chronic medication such as an anti-hypertensive.

Immunizations.  Entering immunizations into the PHR proved to be confusing. One beneficiary searched for tetanus, which resulted in a list of ten options from which to select. After selecting one choice, the “Add Immunization” page allows additional information, such as the reason for the immunization and the date of the appointment, to be entered. This beneficiary was confused by the option to enter “activity time” and “end time,” since receiving an immunization occurs at a specific time and was not seen as an event that had a duration.

Wallet Card.  The Wallet Card page was well-liked by beneficiaries, and some cited it as the most useful function of MyPHRSC. Several participants came to the user observation session with their Wallet Cards in their purse or wallet. They found the information on the card useful to take with them to their clinician, and thought that the emergency contact information was especially important to carry around. Despite their overall appreciation of the Wallet Card, participants offered a few suggestions for improving this function.

Firstly, the Wallet Card page did not retain information that the beneficiary had previously entered. Beneficiaries were therefore required to reenter these details each time they wanted to create a Wallet Card. Secondly, beneficiaries were, at times, unclear about the meaning of the terms used in the Wallet Card feature. The first open field of the Wallet Card is “Provider.” One beneficiary did not know what that meant, one was unsure of which doctor to enter, and another thought the PHR should automatically populate the field with the primary care physician. Thirdly, participants took issue with the way the Wallet Card presented information. One participant was unhappy about how the medications appeared on the card, because it did not include the frequency with which the medications were taken. The beneficiary created a work-around by re-entering the entire medication list in the “Other” field under the “Known Allergies” heading, so that the desired details would be included. The beneficiary did not enter it in the actual “Other Information” section (with heading), because this section did not print out within the form. Finally, a couple of participants critiqued the Wallet Card as being too big when printed. It did not fold up to easily fit within a wallet.

Health Calendar.  The Health Calendar page allowed beneficiaries to add activities in preselected categories offered by the MyPHRSC tool. The categories were: health issue, health journal, immunization, procedure, and visit. For the health issue, immunization, and provider categories, beneficiaries picked the applicable event from a provided list of options. For instance, when adding an immunization event, beneficiaries first selected the immunization option from a drop-down list and then would be directed to another page to choose from the various immunizations listed. The PHR also allowed users the option of filtering the calendar to show only events by specific categories.

The Health Calendar was not a popular function of MyPHRSC because beneficiaries viewed it as redundant and less convenient to use than their (usually paper-based) calendar. A few participants had calendars in their kitchens that included the schedule of all their activities, rather than merely medical events. These calendars more easily allowed them to check for scheduling conflicts. One participant, already entering activities into two electronic calendars (Microsoft Outlook and Quicken), explained that updating a third calendar would be burdensome. One beneficiary suggested the online Health Calendar was less convenient because the beneficiary logs into the Internet only once a day. Another suggested that this function could increase its value if the PHR sent beneficiaries alerts or reminders ahead of appointments.

"It's better than the pieces of paper in the drawer."


Claims.  The Claims page provided beneficiaries with a great deal of useful information. Some beneficiaries were familiar with MyMedicare.gov and used the Website to view information similar to that offered in the PHR Claims page. Beneficiaries were glad to see the information from the MyMedicare.gov site included on the Claims page. However, this page also evoked some questions that were similar to those that arose about other pages (e.g., unknown health providers or facilities), as well as some financial questions. For instance, the amount listed in the “Total Billed” column on the Claims page was at times smaller than the amount in the “Total Paid” column.

Authorized Representatives.  Few participants had used the Authorized Representative page prior to the user observation. Many indicated that they would add their children as authorized representatives to the MyPHRSC tool when they returned home. One beneficiary mentioned being willing to give access to a son; however, because the son would not access the PHR frequently, and with password changes every few months, the beneficiary wondered how useful this would be. Others mentioned that they would be willing to give access to their health providers if the providers thought it would be valuable to them.

"I probably wouldn't go into a password protected site to search for information."


Healthwise and Trusted Websites.  When beneficiaries first viewed the Healthwise page, they thought it could provide a useful resource. However, the searching process was difficult for some. For instance, when a beneficiary clicked on a letter to view a list of all topics beginning with “H,” a “content error” message appeared. After trying “hypertension,” a long list appeared with everything from “Pregnancy and Childbirth” to “Pseudotumor Cerebri.” Another beneficiary tried searching for “Compression Fracture — Spine" and received no results. When the search was altered to “compression fracture,” a long list appeared, but nothing related to spinal injuries. In general, beneficiaries felt that searching for information using other online resources such as Google.com was more useful and convenient.

MyPHRSC also includes a Trusted Websites page that features links to relevant sites. These links are organized in categories such as “Men’s/Women’s Health”, “Disease Specific Resources”, “South Carolina Resources” and “Social Security”. Some participants thought the list of Trusted Websites was useful and had visited some of the pages through other sources. One thought it was good to have a list of websites that had been vetted, so beneficiaries could be assured that the information on those sites would be useful and not commercial.

Health Assessment: My Family History and My Social History.  As a whole, the Health Assessment functionality appeared to be user-friendly. Most participants had either completed or thought that completing the My Family History section would be useful. One participant was disappointed that the option for listing relatives’ conditions did not include a category related to heart disease. The page did not allow users to input additional conditions other than the fewer than twenty options given. Few beneficiaries had used the My Social History section. One participant was not impressed with the half dozen pre-set questions that it offered (related to tobacco, alcohol, and drug use, seat belt use, and exercise) and wished that there were space to enter additional information beneficiaries considered relevant to their social history.

View full report


"report.pdf" (pdf, 1.89Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®