Vignettes
Addressing the Opioid Crisis through Better Data and Data Infrastructure
This vignette provides examples of how a range of OS PCORTF cross agency projects are working to address the opioid crisis by: 1) improving the quality and timeliness of outcomes data, 2) increasing collection of patient-reported information, and 3) building linkages to address co-morbid conditions that affect patient outcomes.Generating Real-World Evidence by Strengthening Real-World Data Sources
This vignette provides examples of how a range of OS PCORTF cross agency projects are working to strengthen the availability of Real World Evidence by standardizing Real Word Data sources so they are fit for use and by building linkages across sources so data are easier to share and analyze.Building the Data Capacity for Patient-Centered Outcomes Research: 2019 Vignettes
This report presents brief vignettes that describe highlights of projects that are funded through the OS-PCOR trust fund. The report gives illustrative examples of the value of the work that has been done in partnership with HHS agencies to build data capacity for patient-centered outcomes research. The vignette document showcases accomplishments of the portfolio and explains the usefulness of the products that project teams are producing.
Project Spotlights
Standardizing Narrative Text for Public Health Research
This spotlight highlights a CDC project - Development of a Natural Language Processing (NLP) Web Service for Public Health. This project developed an NLP Web Service or workbench, called the Clinical Language Engineering Workbench (CLEW). CLEW is a cloud-based web service that hosts NLP and machine learning tools that researchers can use to convert unstructured clinical data into standardized coded data. Once converted, data can be analyzed for public health and clinical research and surveillance.Data Linkage Helps Researchers Find Patterns in Opioid Use and Treatment Response among Patient’s Hospital Encounters
This spotlight highlights the National Center for Health Statistics (NCHS) within the Centers for Disease Control and Prevention (CDC) three projects to address these data infrastructure needs. Since 2017, NCHS has integrated multiple data sources to build innovative data resources that will improve researchers’ ability to analyze data on hospital care, co-occurring mental health and substance use issues, post-acute care delivery, and deaths related to opioid-involved overdose. Through utilization of these integrated data resources, researchers can gain insight on patient characteristics, patterns of care, and other risk factors that affect health outcomes in cases of opioid misuse or abuse.
Infographics
2019 Portfolio Infographic
This infographic depicts the unique features of the portfolio in 2019 including including collaborative federal agency-led work, building data capacity for patient centered outcomes research, and the ability to respond and pivot to address pressing national policy priorities.
Annual Reports of the OS PCORTF Portfolio
2019 Portfolio Report Full Report
The OS PCORTF Annual Report provides project descriptions for each of the OS-PCORTF portfolio’s 27 projects that were active in calendar year 2019, as well as the major accomplishments.2019 Portfolio Report Executive Summary
The Executive Summary provides a synopsis of the portfolio accomplishments for 2019.Building the Data Capacity for Patient-Centered Outcomes Research: The 2018 Annual Report
The OS PCORTF Annual Report provides project descriptions for each of the OS-PCORTF portfolio’s 25 projects that were active in calendar 2018.2017 Annual Report of HHS Projects to Build Data Capacity for Patient-Centered Outcomes Research
The OS PCORTF Annual Report provides project descriptions for each of the OS-PCORTF portfolio’s 21 projects that were active in calendar year 2017.2016 Annual Report of HHS Projects to Build Data Capacity for Patient-Centered Outcomes Research
The OS PCORTF Annual Report provides project descriptions for each of the OS-PCORTF portfolio’s 18 projects that were active in calendar year 2016.HHS Projects to Build Data Capacity for Patient-Centered Outcomes Research: Completed Projects FY 2010 through FY 2015
The purpose of this document is to provide a summary of projects that were funded through the Office of the Secretary Patient-Centered Outcomes Research Trust Fund (OS-PCORTF) and completed between fiscal years (FY) 2010 and 2015.
Project Reports
Enhancing Data Resources for Studying Patterns and Correlates of Mortality in Patient-Centered Outcomes Research: Project 1 - Adding Cause-Specific Mortality to NCHS’s National Hospital Care Survey by Linking to the National Death Index
This final report summarizes the FY 17 OS PCORTF project from the Centers of Disease Control and Prevention (CDC), that linked National Hospital Care Survey to the National Death Index to obtain cause-specific mortality. Cause-specific mortality is a key patient-centered outcome because it allows deaths unrelated to care to be excluded.Emergency Medicine Opioid Data Infrastructure: Key Venue to Address Opioid Morbidity and Mortality (Capturing Opioid Use Disorder Electronically and Patient Reported Outcomes)
This report summarizes the FY 18 OS PCORTF Project managed by the National Institutes of Health/National Institute of Drug Abuse to enhance capacity to use electronic health record (EHR) data and patient reported outcomes measures (PROs) to conduct opioid-related research in emergency departments (EDs).Common Data Model Harmonization Final Report
This project was a collaboration between FDA, several NIH Institutes (NCATS, NCI and NLM), and ONC. The goal of the project was to reuse data, methods, and other resources from existing research networks and to take advantage of available open, consensus-based standards and to build reusable data mappings and transformation services. The project resulted in a Health Level Seven (HL7) Fast Healthcare Interoperability Resources (FHIR) CDMH Implementation Guide, a data governance framework, methods and standard process for ongoing curation, validated mappings between the Common data models, a portal and clinical query software, and public access to data specifications and visualizations of the mappings.Office of the National Coordinator’s Privacy and Security Framework Final Report
This project was a collaboration between ONC and CDC. The goal of this project was to develop resources to support the protection of privacy and security of electronic health data as it is used for PCOR. The resources resulting from this project consider the legal and regulatory requirements relative to patient consent, privacy, and autonomy in examining the factors of collection, access, use, and disclosure of electronic health data that were current at the time of the project. The project resulted in a framework and testing of technology that support the use of health data from a wide variety of sources for PCOR.Women’s Health Coordinated Registry Network Final Report
The project was a collaboration between FDA, NIH/NLM, and ONC. The project produced several publically available products including the Stress Urinary Incontinence (SUI) Surgery Module aimed at collecting quality data on all types of SUI surgeries, WHT-CRN Common Core Dataset; and an HL7 WHT-CRN FHIR Implementation Guide, which is a “recipe book” that provides general guidelines for how the implementation of the FHIR standard can be applied to support the capture and exchange of data among the registries.Developing a Strategically Coordinated Registry Network for Women’s Health Technologies: Office of the National Coordinator
This report summarizes the results of ONC’s work on an interagency project to develop and test a standards-based approach to establishing a new Coordinated Registry Network for women’s health technologies and to develop tools that facilitate data collection to populate registries. The ONC portion of the project resulted in the development and testing of a Health Level Seven International (HL7®) Fast Healthcare Interoperability Resource (FHIR) Implementation Guide that provides guidance regarding the capture and exchange of women’s health data by registries.Enhancing Data Resources for Studying Patterns and Correlates of Mortality in Patient-Centered Outcomes Research: Project 4 - NDI Workshop and Strategy Paper
This report summarizes the barriers to the national death index (NDI) from the Federal and State perspective, as well as the long term strategy for NDI - on how access and use of NDI data may be improved for research. The strategy examined altering the economic model used to support NDI; de-identifying and minimizing the non-economic barriers to accessing and using NDI; re-using cause of death data (e.g., sharing the data with other research proposals; multiple use of approved data for other research studies); improving the efficiency of the administrative aspects of linkages; improving the timeliness and quality of the NDI data; and improving the timeliness of the NDI approval processData Quality Metrics System Final Report
This Final Report is for the Food and Drug Administration’s Data Quality Metrics Project. This project designed, tested, and released for open-source use a web-based data quality toolkit for exploring and describing the quality, completeness, and stability of data sources and visualization of data quality metrics from any data source.Advancing the Collection and Use of Patient-Reported Outcomes through Health Information Technology: Final Report
This report describes the project led by the Office of the National Coordinator for Health Information Technology (ONC) and the Agency for Healthcare Research and Quality (AHRQ) focused on standardizing the collection, exchange, and integration of patient-reported outcome (PRO) data in electronic health record (EHR) systems and other health information technology (IT) solutions to support the electronic sharing of this information.NLP CLEW-Final Report
This report summarized the goals, methodologies, accomplishments and deliverables of the project.NLP CLEW – Lessons Learned
This report compiles lessons that were realized and captured while developing a natural language process (NLP) workbench. The lessons have been separated into three categories that represent overall general observations; tools, systems development, and testing; and NLP and machine learning pipeline and model development.NLP CLEW User Guidance Document
This document explains how to install and use the Clinical Language Engineering Workbench (CLEW) and products developed in the Centers for Disease Control and Prevention (CDC) and the Food and Drug Administration (FDA) pilots.NLP CLEW Workbench Web Service Technical Report. This report presents a detailed technical description of the core NLP approach of the prototype version of the Workbench and two pilot applications developed using the Workbench. In doing so, the core motivating use cases for NLP across a range of clinical domains were explored. The tools and constituent components that are available as part of the prototype are described.
Patient Matching, Aggregation, and Linking (PMAL) Project Final Report
This report explains the project, deliverables, and lessons learned in the areas of improving patient matching, improving data quality, expanding data sharing, and data standardization.Pilot of a Data Quality Framework to Support Patient Matching
This report is based on a pilot performed to test the Data Quality Framework for Patient Matching Framework. The implementation considerations in this paper are intended to assist organizations in implementing the PDDQ Framework (or the Ambulatory Guide) and to encourage their contributions to developing additional resources to improve the quality of demographic data and patient matching.Source Data Capture From EHRS: Using Standardized Clinical Research Data
This report describes the methods and tools to automate the flow of structured electronic health record data into external systems and thereby reduce operating costs, save time, and improve data quality for clinical trials. It is based on a collaboration between investigators at the University of California San Francisco (UCSF) and FDA.-
Improving the Timeliness and Quality of State Electronic Death Registration Systems
The CDC/National Center for Health Statistics (NCHS) worked with jurisdictions in 19 states to address three objectives. The project worked to strengthen both the states’ mortality data infrastructures and the NCHS infrastructure for more timely delivery of state records the National Death Index (ND) database; conducted intersystem exchanges between EHRs and electronic death registration system using national standards; and linked the National Hospital Care Survey inpatient and emergency department data with the NDI data to pilot test capturing within and post hospital mortality. Conceptualizing a Data Infrastructure for the Capture and Use of Patient-Generated Health Data
ONC defines patient-generated health data (PGHD) as health-related data created and recorded by or from patients outside of the clinical setting to help address a health concern. The rise of innovative digital health technologies has increased the ease of capturing, using, and sharing PGHD. Although patients are creating an abundance of PGHD, several technical and cultural barriers have slowed the adoption of PGHD in care delivery and research. This report envisions a health IT ecosystem that optimizes PGHD use for care delivery and research settings and developed a Practical Guide that offers suggested practices and questions to consider for the implementation of the capture, use, and sharing of PGHD in clinical and research settings.Creation of LOINC Equivalence Classes
LOINC provides standardized codes and names for a wide range of clinical observations and provides codes for the observations recorded in a spectrum of settings for diverse applications. This project’s purpose was to create a flexible, extensible, and computable mechanism for rolling up LOINC codes into clinically relevant equivalence groups that enable more efficient processing and aggregation of laboratory and other data from diverse health IT systems.Development of Harmonized Outcome Measures for Use in Patient Registries and Clinical Practice: Methods and Lessons Learned
Variation in types and definitions of outcome measures used in patient registries make it difficult to compare, link, and aggregate data across a range of registries even in the same clinical area. This project report describes the use of the Outcome Measures Framework to develop minimum sets of standardized outcome measures in five clinical areas (atrial fibrillation, asthma, depression, lung cancer, and lumbar spondylolisthesis) for use in patient registries and in clinical practice.Enhanced Data Resources for Studying Patterns and Correlates of Mortality in Patient-Centered Outcomes Research
This CMS project links CDC/NCHS National Death Index (NDI) data for known Medicare and Medicaid deaths with Medicare and Medicaid data in the CMS Chronic Conditions Warehouse (CCW). The CCW Medicare and Medicaid data includes claim and enrollment information. Linkage of NDI and CCW selected data were updated into the Master Beneficiary Summary file, the Medicaid Enrollee Supplemental file, and included in the CCW data dictionary.Collection of Patient‐Provided Information through a Mobile Device Application for Use in Comparative Effectiveness and Drug Safety Research
This FDA project developed a mobile device application for use in clinical trials and other research. The pilot test was conducted with pregnant women although the app is applicable to a range of topics and research methodologies. The project is now known as “FDA MyStudies App”. For additional information, please refer to the Appendix. The output of the project is found on the FDA website.Utilizing Data from Various Partners in a Distributed Manner
The purpose of this project, Utilizing Data from Various Partners in a Distributed Manner was to develop and test the capability to conduct rapid and secure distributed regression analysis utilizing data for different people that are held at different institutions (horizontally partitioned). In addition, it explored the feasibility of creating a virtual linkage capability to utilize data about a single person that is held at different institutions (vertically partitioned). The goal of the project was to enable research sites within a distributed research network to maintain control of patient-level data while generating valid regression estimates across the network without the need to transfer sensitive information.Cross-Network Directory Service Project
This project creates an open source interoperable service that allows: 1) data partners to easily participate in multiple data research networks, 2) queries to seamlessly move across such networks, and 3) users to share analytic capabilities and knowledge across networks. This project pilot tested the Cross-Network Directory Service (CNDS) across two existing networks: FDA’s Sentinel and PCORI’s National Patient-Centered Clinical Research Network (PCORnet).Legal and Ethical Framework to Use Centers for Disease Control and Prevention Data for Patient-Centered Outcomes Research
This paper offers a legal and ethical framework to navigate the legal and ethical challenges of allowing CDC’s data to be used for PCOR. It outlines the legal restrictions on CDC to answer the question “what can CDC do to support PCOR?” It also offers the ethical guardrails for CDC to consider the question “what should CDC do to support PCOR?” Both legal and ethical considerations are essential to CDC in this process.
Other Reports
State Data for Conducting Patient Centered Outcomes Research to Improve Maternal Health: Stakeholder Discussions Summary Report
This report presents a summary of discussions with stakeholders conducted to identify key components of successful state-based maternal health data linkage initiatives, to recognize the gaps in implementing such initiatives including data use for research and finally to understand the current status of existing linkages of state-level data with other data sources and with electronic health records.Challenges and Improvements for PCOR Data Infrastructure: Results from a Stakeholder Prioritization Activity
ASPE sought to gather perspectives on challenges and improvements for PCOR data infrastructure from a diverse group of stakeholders—with a wide range of occupational backgrounds including policy, health care delivery, research and informatics—through an online prioritization activity. Participants first generated challenges and improvements for the five functionalities. Participants then voted on challenges and improvements within each functionality, generating a ranked list of the participant-generated ideas.United States Government Accountability Office, Report to Congressional Committees, November 2020 Comparative Effectiveness Research
In this report, GAO reviewed the CER activities that PCORI and HHS have carried out to meet legislative requirements, described how PCORI and HHS allocated funding to CER activities, and described PCORI and HHS efforts to evaluate the effectiveness of their CER dissemination and implementation activities. The report can also be viewed by visiting: https://www.gao.gov/products/GAO-21-61.Executive Summary: Building Data Capacity for Patient Centered Outcome Research: Portfolio Highlights (2016-2019): Impact, Opportunities and Case Studies. The purpose of this report is summarize the impact of the OS PCORFT portfolio in terms of products, tools and lessons learned for future work. The report is based on an assessment of portfolio awards funded from Fiscal Year (FY) 2016 through FY 2019.
Full Report: Building Data Capacity for Patient Centered Outcome Research: Portfolio Highlights (2016-2019): Impact, Opportunities and Case Studies. This report describes the impact in more depth of a subset of the portfolio awards funded from Fiscal Year (FY) 2016 through FY 2019 and considers opportunities for continued advancement of a robust data infrastructure for patient-centered research.
Addressing the Opioid Epidemic with Better Data: An Overview of HHS Priorities and Projects to Expand Data Capacity for Patient-Centered Outcomes Research on Opioids
This report builds on a workshop sponsored by ASPE, titled Addressing the Opioid Epidemic: Harnessing the Power of Data for Patient-Centered Research in December 2018, which showcased projects working to build the data infrastructure for patient-centered outcomes research around opioids.Building Data Capacity for Patient-Centered Outcomes Research in HHS: A Formative Evaluation of 2012-2016 Projects
This report is a formative evaluation of the OS PCORTF portfolio of projects from 2012 to 2016. It summarizes achievements as well as considerations for future work to build data capacity for clinical comparative effectiveness research and patient centered outcome research.