Evaluation of the Personal Health Record Pilot for Medicare Fee-For Service Enrollees from South Carolina. Appendix 7: USER OBSERVATIONS Discussion Guide


Personal Health Record (PHR) Pilot for Medicare FFS Beneficiaries




1.  Introduction — 5 minutes

We want to start by thanking you for coming today and agreeing to share your thoughts.

We are here today because we work for the National Opinion Research Center (NORC) at the University of Chicago and are conducting a user observation study about personal health records sponsored by the Assistant Secretary for Planning and Evaluation.  We have spoken with you before (by discussion or telephone) and are trying to gain insights into how the personal health record (PHR) fit into your home life, relationships with your health providers, and overall healthcare delivery process.

Today, we will be asking you questions in the form of a causal interview.  This entire observation will last about 2 ½ hours, and you will be provided with compensation in the form of a $75 check. If you returned the health booklet, you will receive an additional $75 check, for a total of $150.

It is important to understand that if you choose to participate, all the information you provide will be held in confidence.  Our conversation and observation will remain absolutely confidential, meaning we will not share what you specifically have to say with your health plan, Medicare, or anyone who is not on the project staff.

We would like to record this session so we can accurately capture our conversation. However, if you would prefer, we can take written notes. Also, if you agree, you may ask to stop the recording at any time, and we will turn off the machine.  If you decide to stop taping, we will ask your consent to retain the portion already taped. Is it OK if we audio record this session?

Your participation is entirely voluntary.  You may leave at any time without penalty, and you may choose not to answer any question or participate at any time. Should you choose not to participate this will not affect your Medicare benefits in any way.

This is a quick summary of what we want to do today:

  • We will spend some time talking with you about your personal background and your medical status. We also want to talk to you about what tools you use to manage your information—such as paper and computer—and what technologies you use for personal and healthcare use.
  • We will walk through the features on MyPHRSC and discuss what you find useful and what you do not.

Do you have any questions for us before we get started?

2.    Background - medical, typical day, role of caregivers — 10 minutes

  • What is your typical day like?
  • Daily routine?  Activities that you engage in.
  • Who lives with you?

Medical related tasks that are a part of this routine (exercise, treatment, medications, appointments with providers, etc)

  • What is your current medical overview?
  • Are there family and/or friends involved in helping with your care?
    [[reference journal question 7]]
  • Do they have legal authorization to decide about your care (healthcare power of attorney)
  • What kinds of things do they do to help? 

(Get specific, recent examples of things that the caregivers have done.)

Ask the above general question, and then probe on the below individual items based on their response.

  • Make medical appointments for you?
  • Track your medical-related finances
  • Transport you on medical-related trips (e.g. pharmacy and doctor)
  • Attend doctor’s office visits?  Help select providers?
  • Help or make decisions regarding
  • Help with post-visit compliance?
  • Do independent research into healthy lifestyles, preventative care and/or treatment options?

How have you let other people who help you see your health information?

  • Why did you decide to do this?


3.    Other Health Technology / Comfort with Technology (in general) — 15 minutes

  • What is your favorite piece of technology? What is your least favorite?
  • Probe: Tell me about a situation where technology has failed you / made you uncomfortable 
  • How do they self describe themselves in terms of technology proficiency and comfort (computer or otherwise)?

Do you use the Internet?

  • Where do you access the Internet?
  • Type of connectivity?

How do you use the Internet?

  • What sites do you use?  What do you use the web for? (E.g., Shopping, banking, searching for health information?  Do you use the internet as a general source of information?)
  • Do you go online to find information about your health or healthcare?

Are there other technologies in your household (e.g. DVD, Tivo, Nintendo Wii, cell phone, PDA)

  • For what purposes do you use them (medical and non-medical)? 
  • When do you use them?
  • Why did you decide to use these technologies?  (Get at sources of motivation)

Other health-related technologies (glucometer, other home treatment or diagnostic devices)

  • How did you learn to use these technologies?
  • Where is the information from these devices recorded?
  • Is data from the devices ever reported back to the providers?  (automated, logged by beneficiary on  chart, etc.)
  • Why did you decide to use these technologies?  (Get at sources of motivation)

4.    Health Information Management — 40 minutes

  • The last time you had a health problem or concern, how did you find out more information about that problem or concern?
  • Let’s go over health booklet to discuss how you organize your health information. (Focus on WHY… why do you keep your information here? Do you have a system? Is it categorized, etc.?)
  • Please describe where you keep your medical records (including medical bills, lab results, prescriptions, etc.). For example, you may store them in a file cabinet in your bedroom, or in a drawer in your kitchen.
  • Please describe where you keep your general book-keeping records (e.g. general bills, banking information).
  • Please describe where you keep contact information for your providers (e.g. phone number for pharmacy, address of doctor’s office).
  • Please describe where you keep medical information that you may need to access in the event of an emergency.
  • Please describe where you keep general information regarding your health or medical conditions. For example, if you have diabetes, where do you store information related to the different types of treatment available for diabetic patients?
  • Please describe where you keep medicines that may have been prescribed by your providers and where do you store over-the-counter medications (e.g. in the bathroom cabinet, on the bedside table).        
  • Do you keep any health information on your person (e.g. in a purse or wallet)? If so, what information do you keep and how is it stored?
  • Where do you keep any general health-related information that you use (e.g. health journals, newsletters, books in a family library, magazines)
  • Where in your home is your computer? If you use a computer outside of your home, please describe where that is located.
  • Please explain anything else that explains how you organize your health information.

How accessible is information when it is stored in different locations? When do you use different information?

Is information you use often stored in easier to access locations? What locations are those? What information is that?

What information do you use rarely? Where do you store it?

    • (Based on the participants’ response, the researcher can organize the information, following the discussion, according to the four types of personal information management techniques noted in the JAMIA article: Just-in-time, Just-at-hand, Just-in-case, Just Because.)


Tell us how using MyPHRSC changed the way you keep track of your records.


Tell us about the challenges you face in organizing your health information.

[[Reference Journal, “Challenges” questions]]

  1. Please write about a time when you could not find your health information.

  2. Have you ever lost important information? Have you ever felt frustrated when trying to organize your health information?

  3. What are the major challenges for organizing your health information?

    Tell us about any successes you have found in organizing your health information.

    [[Reference Journal, “Successes” questions]]

  4. Please write about a time when you had all the health information you needed. 

  5. What do you find particularly useful for organizing your health information? Are there specific locations or tools that help you?

  6. Is some information easier to organize than other information?

5.       Current Use of PHR - when, why, who, where; other users; role PHR plays in health management; perceived value; impact on relationship with caregivers and providers — 25 minutes
Have you used other electronic personal health records like MyPHRSC?

  • How often do you use MyPHRSC? Have you used it this month?
  • What locations and computers have you used to access it?

Think about the last time you that you had a health problem or concern that you thought MyPHRSC might be helpful with.  Did you decide to use MyPHRSC to help with this? Why or why not?  How satisfied were you with the help you got from using MyPHRSC [or from whatever else they did]?  If you did not use MyPHRSC to help with this health concern, are there are other health or healthcare issues you think MyPHRSC could be helpful with? 

    • [If they report being satisfied using MyPHRSC above]: Tell me about how this was helpful for you.  Do you think using MyPHRSC had a positive impact on your health or healthcare? 
    • What was appealing that drove you to switch to MyPHRSC?  When did the issue occur?  Is this issue still relevant to you now?

In thinking about your experiences using MyPHRSC, what suggestions do you have that would improve the PHR? What would make this more useful for you?

  • How has the PHR impacted your relationship with the people who assist you with your healthcare? (e.g. quality of communication, level of health educational awareness, etc.)

  • How has it changed your interactions, for better or for worse?

  • How has the PHR impacted your relationship with your provider?

    • Have you shared information from it with a provider?
    • How did you share this information?
    • How would you have previously shared this information?
    • For better or for worse?
    • Has the PHR prepared you to better communicate with your doctor/provider?
    • Has your PHR helped improve your understanding of your medical conditions and medical status?

  • Emotional Affinity: This activity provides a framework which allows participants to talk about some of the emotions they experience when using MyPHRSC. We will use some stock images affixed on two 11 x 17 pages, to get reactions from users on which images most closely related to how they feel about MyPHRSC. 

    • Person speaking to a pharmacist
    • Police officer pulling over a motorist
    • Family eating dinner
    • Book club or other type of discussion
    • Parent talking to child
    • Person ordering fast food
    • Sitting in traffic
    • Person talking to a retailer who offers advice (butcher, hardware store clerk, etc.)
    • Visiting a library

Use these images to tell us about some of the feelings you experience when using MyPHRSC

    • Which photos can you relate to MyPHRSC?
    • Which photos clearly do not relate?

Use these images to tell me how you would like to feel when using MyPHRSC (future emotional desires)

    • Which photos can you relate to MyPHRSC?
    • Which photos clearly do not relate?

  • Would you like to keep using the PHR?  Is this a valuable solution for you, and why?

  • Would you pay a monthly fee to keep using it?  If so, how much?

  • Is there anything that could be changed about the PHR that would make you willing to pay (or pay more) for the PHR? For example, would you be willing to pay for a “concierge” or “coach” service to help you gather your personal health information?

  • Do you know anyone else using a PHR?  How are they using it?  How does it compare to your experience with MyPHRSC?

6.       Key Features, Key Challenges - likes / dislikes, cognitive human factors issues, design, discoverability, navigation, etc — 45 mins

Now we’d like to walk through the features of the PHR and observe you use it. Please talk aloud to explain what’s going through your mind. We are going to start with a couple scenarios and see if you find these features useful.

Questions for moderator to consider throughout the scenarios:

  • Users can also be asked/observed on the ease of use of each feature.
  • Go from easier to harder functions. Look out for user frustration
  • How does the user approach the feature?  What do they talk about when they use the feature?
  • Have the users discuss the value of different features and to prioritize them.
  • How would they tie this feature into their current personal information management practices (offline or online)? Other medical management devices (e.g. glucometer)?

Health Record Summary

  • Ask user to access their health record summary

  • Illnesses and Conditions

  • Ask user to access this page

  • Ask user to add a condition
  • Visit Summary

  • Ask user to access visit summary

  • Medications:

  • Please use the formulary

  • Ask user to enter medications into the system.

  • Wallet Card:
    • Please make a wallet card and show how you would print it out.

  • Provider Search
    • Find a new provider.

  • Health Calendar
    • Pretend like you are about to go to an appointment with the new doctor you found. First, check what time your appointment is on the Health Calendar.

  • Claims
    • Now, you’ve returned from your appointment and want to check the Medicare claims from that visit, and the record of the visit summary.
    • What value do you get from the CMS claims data? Do you currently use myCMS.gov for this same claims data, and how would you contrast the two tools for accessing some of the same data? 

  • Authorized Representative and Permissions:
    • Add an authorized representative and tinker with their ability to see your information.
    • How could this feature be more useful?

  • Healthwise and Trusted Websites
    • Look up information about an ailment. Which resources do you find more useful?

  • Please look at these additional pages:
    • Health Assessment
    • Frequently Asked Questions
    • User Preferences

7.    Changes / Improvements / Unmet Needs / Additions / Concerns (privacy, security, reliability, etc) — 10 mins

  • Do you have any concerns about the availability of the MyPHRSC website, or have you had any problems accessing it?
  • Do you have any concerns about using an electronic personal health record?
  • How do you feel about the privacy and security of your information in MyPHRSC?
  • Would you be more or less likely to use a PHR sponsored by the following, and is this due to privacy/security concerns?
    • CMS
    • Another government agency (state or federal)
    • Your provider
    • Other source

8.    Describe typical health management tasks (in home) - discuss how these activities should link to PHR (e.g. taking medications, vitamins, injections, recording blood pressure in a journal). — 5 mins

  • Is there any current connection between these home health management tasks and the PHR?
  • Is there any way that they are connected?  Should be connected?
  • How has this or could this make tasks better

View full report


"report.pdf" (pdf, 1.89Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®