Below, we present detailed findings from each evaluation activity. We begin by sharing results from background discussions with MyPHRSC contractors. Next, we provide a detailed analysis of each of the qualitative and evaluation activities. This is followed by an overview of pertinent results from the usage data analysis.
Findings are organized as follows:
- Discussions with MyPHRSC Contractors
- Discussion Groups with Beneficiary MyPHRSC Users
- Semi-Structured Interviews with Beneficiary MyPHRSC Nonusers
- Observations of Beneficiary MyPHRSC Users
- Discussion Group with Providers
- Usage Data Analysis
Discussions with MyPHRSC Contractors
NORC conducted telephone discussions with three key informants from companies involved in implementing the pilot program. These discussions, including experiences both before and during this project with respect to development and functionalities of the MyPHRSC product, the data lifecycle, standards, security, outreach and challenges experienced, served to inform NORC’s understanding of MyPHRSC and provided context for conducting further evaluation activities.
Discussion Groups with Beneficiary Users of MyPHRSC
When screening for the user meetings, the selected participants indicated a broad range of experience levels in using computers and in using the PHR. Beneficiaries reported first hearing about the PHR between 1 month and 6 months prior to September, and first signed up for the PHR between 1 week and 4 months prior to September. Expectations of MyPHRSC varied. Many beneficiaries reported that they did not know what to expect of the PHR and wanted to simply learn more about the tool, while some reported signing up in hopes that the PHR could help them to keep track of their records (e.g. billing, claims and personal health information). Appendix 9 lists the results from the pre-discussion screener.
In discussion groups, NORC investigated key components of utility (the quality or condition of being useful) and usability (how well users can learn and use a product to achieve their goals and how satisfied they are with that process). Participants identified computer and Internet experience; key health care priorities; reasons for using the PHR; most valued features and functions; challenges and enablers to using the PHR; key benefits experienced or anticipated; and potential impacts of using the PHR. Below, we provide a summary of key results and conclusions.
Semi-Structured Interviews with Beneficiary MyPHRSC Nonusers
In discussion groups, beneficiary nonusers of MyPHRSC identified computer and Internet experience; key health care priorities; reasons for registering for MyPHRSC; reasons for not using the PHR; suggestions for PHR developers; and comments regarding the usability and utility of MyPHRSC. Below, we provide a summary of key results and conclusions.
Laboratory Observations of Beneficiary Users of MyPHRSC
Discussions held one-on-one with beneficiaries were focused on understanding beneficiary backgrounds, comfort with technology, health information management practices, their use of MyPHRSC, how this integrated into their normal daily workflow and some of the issues/challenges that users had encountered when interacting with the PHR. Watching beneficiaries navigate through MyPHRSC revealed the successes and challenges they have with using different features.
Beneficiaries found MyPHRSC to contain useful medical information and viewed it as a helpful tool for better managing their health. Below is a summary of the key observations that emerged over the course of the user observations.
Discussion Group with Providers
Discussion topics for the meeting with providers covered provider experiences using health information technology, provider understanding of PHRs, challenges and enablers using PHRs, perceived impact of PHRs to provider workflow and suggestions for future PHR development. Below we provide a summary of key results and preliminary conclusions.
The results of the pre-discussion screener indicate that participating providers had a broad range of experience using health information technology. Three providers knew about MyPHRSC prior to November; three providers reported using EMRs in their practices; and one provider reported having patients who used PHRs. Providers who participated came from a variety of practice types in locations throughout South Carolina, with most providers working in family practice and internal medicine, as well as one chiropractor and one optometrist. Provider experience varied from 12-38 years in practice; with the majority of providers practicing for 18 years or longer. The results of the pre-discussion screener are included in the Appendix 11.
Analysis of Usage Data
This analysis of the usage data provided quantitative information that contributed to answering the key research questions of the evaluation. This section provides descriptive statistics about variables of interest, and then presents multivariate results for the three study hypotheses.
We provide an overview of the total South Carolina Medicare population, the population targeted for the purposes of this pilot, and the users registering for MyPHRSC as well as those that registered and logged in more than once into the tool. In the following section, we describe the characteristics of the registrants and users.
Figure 1. Relationship between South Carolina Medicare Beneficiaries, Registrants, and Users in the MyPHRSC Pilot.
Figure 1 provides an overall snapshot of the number of South Carolina Medicare beneficiaries who took advantage of the MyPHRSC tool.