PCORnet® 2020: current state, accomplishments, and future directions.

OBJECTIVE: To describe PCORnet, a clinical research network developed for patient-centered outcomes research on a national scale. STUDY DESIGN AND
SETTING: Descriptive study of the current state and future directions for PCORnet. We conducted cross-sectional analyses of the health systems and patient populations of the 9 Clinical Research Networks and 2 Health Plan Research Networks that are part of PCORnet.
RESULTS: Within the Clinical Research Networks, electronic health data are currently collected from 337 hospitals, 169,695 physicians, 3,564 primary care practices, 338 emergency departments, and 1,024 community clinics. Patients can be recruited for prospective studies from any of these clinical sites. The Clinical Research Networks have accumulated data from 80 million patients with at least one visit from 2009 to 2018. The PCORnet Health Plan Research Network population of individuals with a valid enrollment segment from 2009 to 2019 exceeds 60 million individuals, who on average have 2.63 years of follow-up.
CONCLUSION: PCORnet's infrastructure comprises clinical data from a diverse cohort of patients and has the capacity to rapidly access these patient populations for pragmatic clinical trials, epidemiological research, and patient-centered research on rare diseases.

PCORnet is a national-scale clinical research network with standardized, analysis-ready EHR data for 80 million Americans and the ability to conduct large-scale pragmatic research on health problems facing the nation.

Introduction

Overview

In 2013 the Patient-Centered Outcomes Research Institute (PCORI®) announced that it would fund the development of PCORnet®, the National Patient-Centered Clinical Research Network (pcornet.org). The guiding vision was the formation of a national network-of-networks that engages patients, caregivers, clinicians, health system leaders, payers, and researchers in the design, conduct, and advancement of patient-centered outcomes research [1]. PCORnet® Network Partners would form a national infrastructure that could simultaneously support observational studies using electronic health records and health plan data while also conducting pragmatic clinical trials embedded within routine care settings. Both types of studies would generate new evidence that was timely, meaningful, and useful [2,3]. The infrastructure supporting these activities would include institutional leadership leveraging a novel collaboration platform comprising comprehensive clinical data that is standardized, analysis-ready, and derived from medical institutions and health plans, common network and data governance, streamlined contracting and regulatory agreements, and resources for deeply engaging patients [4]. A large-scale consortium (i.e., a network-of-networks) with common administrative, technical, and governance resources and established health system partnerships would make research start-up, patient recruitment, and creation of large data-sets for observational research faster, easier, and more efficient than the status quo.

At its launch, the PCORnet infrastructure included a Coordinating Center, Clinical Research Networks composed of healthcare organizations, and Patient-Powered Research Networks led by patients and patient organizations [1]. As PCORnet has matured, the number of participating Network Partners has fluctuated, the Patient-Powered Research Networks have moved to a free-standing status, and Health Plan Research Networks have been added. Investigators who use the PCORnet infrastructure come from within and outside of Network Partner institutions.

Stakeholder engagement

PCORnet was developed on a bedrock of patient-centeredness, which is fundamental to all research funded by PCORI [3]. Patients are engaged in PCORnet’s governance and research project leadership, conduct, and dissemination of results (HC and NAW are patient partners and co-authors of this manuscript). Partnering with stakeholders (i.e., patients, caregivers, clinicians, health system leaders, payers, and patient organizations) from the planning phase through the dissemination of research findings helps to ensure that the evidence generated by these studies is meaningful—that is, information that is useful for health- and healthcare-related decision-making and likely to be used. [5] This is done by ensuring that research questions and outcomes are relevant to the ultimate end-users of results [6], enhancing study recruitment using technology tools [7], ensuring that the conduct of research puts participants at the center of all decisions, and including stakeholders in the interpretation, writing, and multi-media dissemination of research findings [5].

Engaging patients and other stakeholders in each phase of the research is a requirement for all studies conducted using PCORnet infrastructure, including clinical trials, surveys, and retrospective data analyses. Engagement teams affiliated with PCORnet’s Network Partners provide resources to help researchers identify stakeholder partners, train and support them, and establish productive, collaborative, and trusting patient-researcher relationships. Network Partners have developed a variety of resources to aid with these engagement activities, such as a toolkit for implementing Community Engagement Studios that enable panels of community stakeholders to provide input to researchers at the study planning phase [8]; a story archive that amplifies patient and caregiver perspectives and brings together stakeholders and researchers with shared interests to form engaged research teams [9]; workshops to develop a shared understanding on how community engagement methods inform but may differ from patient engagement strategies [10]; training materials that help stakeholders participate effectively in research teams [11]; and, various communication approaches that support prioritization of research topics [12,13]. PCORnet is committed to developing the science of stakeholder engagement, using validated engagement tools, sharing effective processes, and scientifically evaluating both.

Objectives

The purpose of this manuscript is to take stock of PCORnet after several years of development. We present the calendar year 2020 snapshot of the current state of PCORnet’s infrastructure, including organization and governance, data, and patient populations. We highlight some of the research studies conducted using PCORnet resources to illustrate the types of scientific inquiry for which the network-of-networks is well-suited. The manuscript concludes with some future directions for PCORnet.

Materials and methods

PCORnet governance

The PCORnet network-of-network is governed by a 16-member Steering Committee composed of 1 representative from each of the 11 networks, two from the Coordinating Center, and, importantly, 3 patient representatives. All strategic and policy decisions are made by the Steering Committee. The Steering Committee is led by an elected chair and vice-chair. An Executive Management Team—led by the Steering Committee chair and including the vice-chair and one representative from the Coordinating Center, Health Plan Research Networks, and patient representatives—prioritizes topics for the Steering Committee. A representative of PCORI participates as a non-voting member in both the Steering Committee and Executive Management Team. A data workgroup makes recommendations on data quality, data and query transparency, security and privacy, and evolution of the PCORnet Common Data Model. Although institutions participating in PCORnet research are encouraged to be part of a single institutional review board (IRB), reliance on a single IRB is voluntary.

Networks and Network healthcare organizations

The list of PCORnet network components is shown in Table 1 . Most of the medical institutions--defined as a healthcare organization with a unique tax identifier--are academic medical centers. Except for organizations participating in the network called “ADVANCE,” which is composed primarily of community health centers, nearly all medical institutions participating in PCORnet Clinical Research Networks are integrated delivery systems with one or more hospitals, outpatient clinics (primary and specialty care), and emergency departments. Across the nine Clinical Research Networks, there are 251 institutions that are organized into 61 data contributors—Table 2 . A data contributor manages the PCORnet data mart for one or more institutions. Overall, PCORnet Clinical Research Networks currently include 337 hospitals, 169,695 physicians, 3,564 primary care practices, 338 emergency departments, and 1,024 community clinics serving medically underserved populations. These healthcare institutions and their clinicians serve as a diverse set of clinical trial sites for pragmatic research conducted in everyday clinical care settings. The two Heath Plan Research Networks, HealthCore and PRACnet, are research subsidiaries of two large national insurance plans, Anthem and Humana, respectively.

Table 1

PCORnet’s Network Components: 2020

Network Component	Description
Coordinating Center	• Led by the Duke Clinical Research Institute and Harvard Pilgrim Health Care Institute; manages the distributed data infrastructure and provides project management resources.
Clinical Research Networks
ADVANCE (advancecollaborative.org)	• A national network of community health centers. • Led by OCHIN, which is based in Portland, Oregon.
CAPriCORN (capricorncdrn.org)	• A network of Chicago area medical centers. • Led by Northwestern University.
GPC (Greater Plains Collaborative)(gpcnetwork.org)	• A network of medical institutions in the mid-west, Texas, and Utah. • Led by University of Kansas.
Insight (insightcrn.org)	• A consortium of New York City medical centers. • Led by Weill Cornell Medicine.
OneFlorida (onefloridaconsortium.org)	• A regional network of medical institutions in the state of Florida. • Led by University of Florida.
PaTH (pathnetwork.org)	• A consortium of medical institutions in the US Mid-Atlantic region. • Led by the University of Pittsburgh.
PEDSnet (pedsnet.org)	• A national network of free-standing children’s hospitals. • Led by the Children’s Hospital of Philadelphia.
REACHnet (reachnet.org)	• A consortium of medical institutions in Louisiana and Texas. • Led by the Louisiana Public Health Institute.
STAR (starcrn.org)	• A network of medical institutions primarily in the southern region of the US. • Led by Vanderbilt University Medical Center.
Health Plan Research Networks
HealthCore (healthcore.com)	• A research subsidiary of Anthem with access to health plan data for patients residing in 14 states.
PRACnet (pracnet.org)	• A health plan network coordinated by Medical Outcomes Management Inc. in partnership with Humana health plan.

Table 2

Clinical Research Networks: 2020

Network name	Data contributors	Medical institutions	Hospitals	Physicians	Primary care practices	Emergency departments	Community clinics
ADVANCE	4	133	0	5,785	498	0	564
CAPRiCORN	8	17	19	10,852	263	19	172
GPC	12	17	78	66,629	551	76	30
INSIGHT	5	7	22	14,124	274	32	33
STAR	8	23	71	18,358	677	70	18
OneFlorida	6	13	47	10,898	339	44	39
PaTH	7	8	58	22,868	608	44	58
PEDSnet	7	7	9	10,230	109	18	49
REACHnet	4	26	33	9,951	255	35	61
Total	61	251	337	169,695	3,574	338	1,024

Distributed data network

The primary data source for the PCORnet Clinical Research Networks is EHR data. An important limitation is that extant data are from health systems, and for some outcomes (e.g., myocardial infarction), there is incomplete ascertainment. PCORnet addresses this potential bias by linking the EHR data to health plan data to obtain complete capture of outcomes. Some data contributors maintain linked data sources, while others have regulatory agreements that enable linkage on a project-specific basis. About two in three data contributors are able to conduct research with linked Medicare or Medicaid data; about one in two can link to private insurance claims, clinical registries, social determinants of health, and death records; nearly three in four can link to tumor registries, and just one in five can link to birth records.

Each data contributor retains its data locally, creating a large-scale national distributed data network. Network Partners transform EHR data or health plan data to the PCORnet Common Data Model [14], which is updated on an annual basis. Data are obtained from inpatient, outpatient, emergency department, and ancillary service settings and across time, creating comprehensive and longitudinal patient-level records of all interactions with member health systems. Data are organized as demographics, vital status, insurance status, vital signs, encounter and provider characteristics, anthropometric measurements, diagnoses, location, drug exposure (prescribed and dispensed), procedures performed, laboratory test results, and primary care, specialty, and acute care (emergency department and inpatient) utilization at institutions within Network Partners. PCORnet Network Partners can also collect data on patient-reported outcomes, such as the PROMIS measures (see healthmeasures.net), and both individual and area-level social and behavioral determinants of health.

Source data from Network Partners are extracted quarterly from clinical information systems and undergo structural data quality assessments that evaluate data against a series of required and investigative data checks. These checks translate into more than 1,500 different assessments that are used to examine the structure of the data, addressing missingness, conformance to the PCORnet Common Data Model or to standard reference terminologies (e.g., LOINC for laboratory test results and RxNorm for prescribed medications), and whether a given record contains enough metadata to be analytically useful (e.g., laboratory records with a result and a corresponding result unit) [15]. Quarterly data extracts that do not meet the required data quality standards are not included in the distributed data network production environment until they pass data characterization. For the majority of PCORnet Clinical Research Networks, EHR data has a latency of three months or less from the date of extract. Institutions within PCORnet Clinical Research Networks provide a tremendous volume of records, with over 14 billion diagnoses, 2.6 billion medication orders, and 9.8 billion laboratory results.

Statistical queries of distributed data network

From January to March 2020, the PCORnet Coordinating Center executed a query of all 61 Clinical Research Network data contributors to describe their combined patient population. All patients with at least one inpatient or outpatient encounter with a recorded diagnosis from January 2009 to December 2018 were included. It is important to note that the query did not de-duplicate patients, so a single patient visiting more than one medical institution participating in PCORnet could be represented multiple times within patient counts. The query was generated and distributed to the Network Partners from the central Coordinating Center using PopMedNet [16], which provides a secure means for sending queries and receiving results from the distributed PCORnet network. Queries were built in the SAS format, and results were collected and aggregated by the PCORnet Coordinating Center.

In April 2020, the two Health Plan Research Networks queried their PCORnet Common Data Model environments. All patients with at least one valid health plan enrollment segment from January 2009 to December 2019 were included. As before, the query did not de-duplicate patients across the health plans or across participating medical institutions.

Results

Clinical Research Network partners’ patient population

By the end of 2018, the PCORnet Clinical Research Network population totaled 80 million patients (Table 3 ). This cohort continues to grow with about eight million new patients added each year. It can be used for observational research using cross-sectional and longitudinal study designs. In the year 2018, there were 29,475,756 patients (37% of the total) who had one visit.

Table 3

Demographic characteristics of the PCORnet Clinical Research Network patient population, 2009-2018

Total number of patients	79,665,703
Age at first visit, years, n (%)
0-17	24,020,116 (30)
18-24	7,145,838 (9)
25-44	19,485,973 (24)
45-64	18,489,378 (23)
65+	10,524,398 (13)
Gender, n (%)
Male	36,499,149 (46)
Female	43,144,820 (54)
Other/missing	21,724 (<1)
Race, n (%)
White	49,649,780 (62)
Black/African-American	11,884,360 (15)
Asian	2,085,901 (3)
Native Hawaiian/Pacific Islander	207,078 (<1)
Other/Missing	15,838,584 (20)
Ethnicity, n (%)
Hispanic	9,869,046 (12)
Non-Hispanic	56,412,402 (71)
Missing	13,384,255 (17)
Year of first visit, n (%)
2009	5,746,179 (7)
2010	8,678,599 (11)
2011	7,136,282 (9)
2012	7,805,083 (10)
2013	8,895,516 (11)
2014	8,648,274 (11)
2015	8,463,646 (11)
2016	8,358,360 (10)
2017	8,041,474 (10)
2018	7,892,282 (10)
Network, n (%)
ADVANCE	4,622,683 (6)
CAPRiCORN	7,702,843 (10)
Greater Plains Collaborative	16,006,105 (20)
INSIGHT	8,649,172 (11)
OneFlorida	6,261,635 (8)
PaTH	10,698,489 (13)
PEDSnet	6,172,582 (8)
REACHnet	6,467,118 (8)
STAR	13,085,076 (16)

The demographic distributions between the 10-year cohort and the 2018 cohort were similar. In general, patients seen more recently are more likely to be available to enroll in clinical studies. By developing trial selection criteria as queries developed in the SAS or SQL and using the common data model, investigators can identify consistent cohorts across the entire network to support the more accurate trial design and execution of recruitment strategies. After networks and their sites complete a query against data obtained from patients seen in the past 18 months, they can be reidentified by the health systems (which hold reidentification keys), enabling a variety of techniques for rapidly recruiting participants into trials. This is a unique ability of the PCORnet infrastructure and has been used successfully in ongoing clinical trials, such as ADAPTABLE, which is studying the optimal dosage of aspirin for secondary prevention of ischemic heart disease [17].

About one in three patients entered the PCORnet Clinical Research Network cohort as children. PEDSnet, the only PCORnet Network Partner devoted exclusively to pediatric research, comprises 25% of 0–17-year-olds in PCORnet, and a much higher share of children with rare diseases. Compared with the 2018 US census bureau statistics [18], the PCORnet Clinical Research Network cohort is skewed toward more females. Recorded race and ethnicity proportions are about the same in PCORnet as the US population. The average sample size per Network Partner is 8.8 million, with a range from 6.1 to 16.0 million.

Within the Clinical Research Network cohort of 80 million patients, from 2009 to 2018, about one in five patients were hospitalized, four in five had an outpatient visit, and three in 10 had an ED visit (Table 4 ).

Table 4

Utilization, lab tests ordered, prescribed medications, and health conditions of the PCORnet Clinical Research Network patient population, 2009–2018

Total number of patients	79,665,703
Utilization, n (%)
Hospitalization	14,543,616 (18)
ED visit	24,994,038 (31)
Ambulatory visit	64,082,627 (80)
Lab tests ordered, n (%)
Creatinine	30,154,275 (38)
Hematocrit	30,949,991 (39)
Hemoglobin A1C	10,263,446 (13)
Low density lipoprotein	14,467,005 (18)
Prescribed Medications, n (%)
Anti-diabetic agents	4,060,505 (5)
Biologics used to treat autoimmune disorders	309,134 (1)
Serotonin reuptake inhibitors (SSRIs/SNRIs)	7,203,303 (9)
Opioids	17,227,324 (22)
Lipid lower agents	7,214,144 (9)
Health Conditions, n (%)
Asthma	5,427,113 (7)
Cancer	5,000,064 (6)
Dementia	1,434,594 (2)
Depression	6,234,432 (8)
Epilepsy	1,109,139 (1)
Heart failure	2,127,304 (3)
Ischemic heart disease	4,112,855 (5)
Osteoarthritis	5,270,586 (7)
Stroke	1,718,203 (2)
Type 2 diabetes mellitus	5,826,597 (7)

Of the 10 common chronic conditions reported in Table 4, depression occurred most frequently, while millions of patients were affected by each of the other conditions. Individuals 65 years old and older, who comprised 13% of the population, accounted for 42% of cancer, 71% of dementia, 61% of health failure, 57% of ischemic heart disease, 45% of osteoarthritis, and 57% of stroke patients. About six million patients had Type 2 diabetes mellitus, but over 10 million had a hemoglobin A1C test, indicating its use as a diagnostic, as well as management tool. Moreover, just four million patients had at least one prescription for insulin or an oral hypoglycemic, suggesting that some patients with Type 2 diabetes mellitus are being treated with lifestyle management.

Health Plan Network partners’ patient population

The PCORnet Health Plan Research Network population totals over 60 million patients (Table 5 ). A strength of the health plan data is the ability to collect information from across health systems both within medical institutions within Network Partners and beyond. Overall, patients are observed for 2.63 years with a range by age group of 1.92 (18–24 years old) to 3.59 (65+ years old), providing for complete person-time ascertainment for observational research, which supports cross-sectional and longitudinal study designs and pragmatic capture of clinical outcomes for trials such as ADAPTABLE [19]. Health plan data can also be utilized to recruit members into pragmatic clinical trials [20]. Each year about four million new patients are added to PCORnet’s Health Plan Research Networks.

Table 5

Health Plan Research Network demographic characteristics, 2009–2019

Characteristic	Total numberof patients (n/%)	Average follow-up time (years)
Overall	61,663,411 (100)	2.63
Age at enrollment, years
0–17	13,170,749 (21)	2.58
18–24	6,395,547 (10)	1.92
25–44	18,289,698 (30)	2.30
45–64	16,307,390 (26)	2.89
65+	7,500,173 (12)	3.59
Gender
Male	30,525,945 (50)	2.59
Female	31,137,466 (50)	2.67
Year of enrollment
2009	17,527,238 (28)	3.52
2010	3,767,918 (6)	2.84
2011	3,455,813 (6)	2.65
2012	3,000,045 (5)	2.64
2013	7,367,423 (12)	3.39
2014	5,758,767 (9)	2.65
2015	4,489,088 (7)	2.29
2016	4,445,110 (7)	1.98
2017	4,119,717 (7)	1.60
2018	3,847,250 (6)	1.21
2019	3,885,188 (6)	0.61

Conclusions

Research using the PCORnet infrastructure

The PCORnet infrastructure is well-suited to facilitate the conduct of large-scale clinical trials. The diversity of clinical sites, common governance, and a common set of regulatory agreements enable PCORnet Network Partners to rapidly mount large pragmatic clinical trials, such as the NIH-sponsored PREVENTABLE trial, which will randomize 20,000 people over 75 years of age to statin or placebo to evaluate its effect on cardiovascular outcomes and cognitive decline. PCORnet Network Partners are also conducting the recently launched placebo-controlled HERO trial (Healthcare Worker Exposure Response & Outcomes--heroesresearch.org) that will evaluate whether hydroxychloroquine can prevent or attenuate COVID-19 illness in healthcare workers. This study went from concept to first recruited patient in a matter of just a few weeks.

With 80 million patients’ electronic health record data stored in the Clinical Research Network databases and another 60 million patients in Health Plan Research Network databases, PCORnet comprises a rich set of resources for epidemiological, health services, and translational research on rare diseases (i.e., health conditions that affect fewer than 200,000 individuals in the United States) or rare events (e.g., uncommonly occurring adverse events associated with medications, long-term effects of immunosuppressive agents).

Clinical data from PCORnet Network Partners provides a rich resource for studying healthcare utilization, healthcare processes such as laboratory test orders and medication usage, and health outcomes such as laboratory results (e.g., hemoglobin A1C, low-density lipoprotein, proteinuria), complications, and new diagnoses, both overall, as well as patients with specific health conditions. When doing so, investigators, informaticians, and analysts work together to create code-sets from ICD-9-CM, ICD-10-CM, and SNOMED CT for health conditions, RxNorm for medications, CPT for procedures, and LOINC for lab results. Forming these code-sets, which may include, in some cases, thousands of codes, is an area of special expertise within PCORnet.

The PCORnet Coordinating Center can deploy statistical queries of the network to rapidly return in a matter of days counts and descriptive statistics for patients with particular conditions or cohorts with various selection criteria to evaluate whether a study is feasible but also facilitate surveillance and observational studies for stakeholders. Investigators will be increasingly leveraging the PCORnet infrastructure for rare disease research to answer questions that are meaningful to patients affected by a rare disease, as well as their caregivers and clinicians.

Future directions

PCORnet Network Partners update their data every 3 months. However, at the time of the writing of this manuscript, the network is undertaking a national surveillance project to monitor rates of COVID-19 illness, testing, complications, and correlates. For accomplishing this project, a subset of data related to COVID-19 illness is being updated on a weekly basis, taking advantage of existing procedures for extracting and transforming source data to the PCORnet Common Data Model.

PCORnet Network Partners are embarking on an approach to use the privacy protected record linkage to link patient data from clinical sites within within Clinical Research Networks with claims data from Health Plan Research Networks. Individual sites will leverage software at source data systems to create de-identified patient tokens that will be loaded into a HASH_TOKEN table [14] in the common data model. Research queries will hit the HASH_TOKEN table and return tokens to the PCORnet Coordinating Center for de-duplication and linkage of patient records for specific research projects. Each project will operate under individual, institutional review boards that will govern these linkages, but the PCORnet governance and technology infrastructure is being designed to scale for scores of future projects leveraging linkage.

After several years of maturation, PCORnet Network Partners have a proven track record in both large-scale clinical trials, observational research studies that focus entirely on extant data or link the common data model to prospectively collected patient-reported data, surveillance of health conditions and health services, and feasibility evaluations of potential research studies. Its door is open for researchers, patients, and patient organizations to bring their questions, ideas, and research projects. Over the next 5 years, we envision the PCORnet infrastructure moving from a large, demonstration project to a public utility that is a valuable asset for improving the health and healthcare of all people in the United States, and ultimately in collaboration with international investigators and networks, across the world.

CRediT authorship contribution statement

Christopher B. Forrest: Conceptualization, Methodology, Validation, Formal analysis, Resources, Writing - original draft, Supervision, Funding acquisition. Kathleen M. McTigue: Conceptualization, Resources, Writing - original draft, Funding acquisition. Adrian F. Hernandez: Conceptualization, Resources, Writing - review & editing, Supervision, Funding acquisition. Lauren W. Cohen: Resources, Writing - review & editing, Supervision. Henry Cruz: Conceptualization, Resources, Writing - review & editing. Kevin Haynes: Conceptualization, Resources, Writing - review & editing, Funding acquisition. Rainu Kaushal: Conceptualization, Resources, Writing - review & editing, Funding acquisition. Abel N. Kho: Conceptualization, Resources, Writing - review & editing, Funding acquisition. Vinit P. Nair: Conceptualization, Resources, Writing - review & editing, Funding acquisition. Richard Platt: Conceptualization, Resources, Writing - review & editing, Funding acquisition. Jon E. Puro: Conceptualization, Resources, Writing - review & editing, Funding acquisition. Russell L. Rothman: Conceptualization, Resources, Writing - review & editing, Funding acquisition. Elizabeth A. Shenkman: Conceptualization, Resources, Writing - review & editing, Funding acquisition. Lemuel Russell Waitman: Conceptualization, Resources, Writing - review & editing, Funding acquisition. Neely A. Williams: Conceptualization, Resources, Writing - review & editing, Conceptualization, Resources, Writing - original draft. Thomas W. Carton: Conceptualization, Resources, Writing - original draft, Funding acquisition.