A Digital Architecture for a Network-Based Learning Health System: Integrating Chronic Care Management, Quality Improvement, and Research.

INTRODUCTION: We collaborated with the ImproveCareNow Network to create a proof-of-concept architecture for a network-based Learning Health System. This collaboration involved transitioning an existing registry to one that is linked to the electronic health record (EHR), enabling a "data in once" strategy. We sought to automate a series of reports that support care improvement while also demonstrating the use of observational registry data for comparative effectiveness research. DESCRIPTION OF ARCHITECTURE: We worked with three leading EHR vendors to create EHR-based data collection forms. We automated many of ImproveCareNow's analytic reports and developed an application for storing protected health information and tracking patient consent. Finally, we deployed a cohort identification tool to support feasibility studies and hypothesis generation. There is ongoing uptake of the system. To date, 31 centers have adopted the EHR-based forms and 21 centers are uploading data to the registry. Usage of the automated reports remains high and investigators have used the cohort identification tools to respond to several clinical trial requests. SUGGESTIONS FOR FUTURE USE: The current process for creating EHR-based data collection forms requires groups to work individually with each vendor. A vendor-agnostic model would allow for more rapid uptake. We believe that interfacing network-based registries with the EHR would allow them to serve as a source of decision support. Additional standards are needed in order for this vision to be achieved, however.
CONCLUSIONS: We have successfully implemented a proof-of-concept Learning Health System while providing a foundation on which others can build. We have also highlighted opportunities where sponsors could help accelerate progress.

Introduction

Billions of dollars have been spent in the United States over the past decade to accelerate the adoption of electronic health records (EHRs), and programs like Meaningful Use are enabling the interoperable exchange of standardized information from the EHR.1,2 Many view these developments as a sign that we are creating many of the technical components of a Learning Health System in which every patient interaction with the health care system provides an opportunity to generate data that can be used to create new evidence and knowledge, which in turn can be used to improve clinical practice.3–6 In its most advanced state, a Learning Health System combines comparative effectiveness research (CER) with quality improvement (QI) science to ensure the delivery of new knowledge at the point of care.

There have been some small-scale successes within a single organization or a network to use data from the EHR to inform the cycle of clinical care, improvement and research.7–10 However, the concept of a broadly distributed EHR-based Learning Health System remains more aspirational than operational.11 This stems in part from two technical challenges: designing information systems that need to support clinical care, research, and QI activities concurrently, and having to fulfill the needs of users who are dispersed geographically and across multiple organizations. In addition to the technical challenges, there are numerous societal and scientific barriers to overcome, such as achieving alignment among stakeholders and handling the difference in operational time scales.7,12 As a result, most efforts have focused on optimizing systems to support the use of EHR data either for research8,10,13–23 or for clinical care and QI.

We approached the problem from a whole system perspective by designing a technology architecture that could simultaneously address the needs of the various stakeholders of the Learning Health System—clinicians who want to provide more effective care, improvement experts who aim to create more effective health systems, researchers who wish to accelerate research, and patients and families who desire better information along with the ability to participate in their own care. Our goal was to create a “proof of concept” example by transforming a web-based multicenter QI registry for a serious and rare chronic illness into an “enhanced registry” that could help improve chronic care management, facilitate QI, enable CER, and engage patients and families. The result would be a network-based Learning Health System that addresses some of the barriers that have hindered previous attempts in this area.24

Description of Architecture

Our proof-of-concept architecture was developed in collaboration with the ImproveCareNow Network, a learning network that was founded in 2007 to advance the quality of care and outcomes of children and adolescents with Crohn’s disease and ulcerative colitis, or inflammatory bowel disease (IBD).25,26 Prior to the start of the project (September 2010), the ImproveCareNow Network’s registry, like many others, placed a degree of burden on those who wished to interact with it. Care centers entered data three times: once during the encounter for clinical documentation, a second time when the data were abstracted onto a paper-based case report form (CRF), and a third time when the CRF data were entered into the registry Web forms. This cumbersome data entry process represented a barrier to participation, as centers needed to hire and allocate staff specifically for data collection. The network’s improvement experts faced hurdles in trying to act on the data in the registry, as the generation of analytical reports required significant manual intervention. For researchers, it was unclear whether the registry’s observational data, which was first collected for QI purposes, could also be used to support CER analyses. In addition, the process to identify potential cohorts of patients for clinical trial recruitment was cumbersome and laborious. Finally, for patients and families, there was no way for them to view the child’s data or to contribute information themselves.

With funding from the Agency for Healthcare Research and Quality’s (AHRQ’s) Enhanced Registry program, we designed a technical architecture that aimed at overcoming the burdens that limited the ability of clinicians, patients, and researchers to participate and contribute to the Learning Health System. This architecture is illustrated in Figure 1, and its individual components are discussed further in the Implementation section below.

Figure 1.

Functional Architecture of the Learning Health System that Supports the ImproveCareNow Network

Note: The components that support data input are listed on the left and the components that support research and analytics on the right. All tools are accessible from the same user interface. The labeled ovals correspond to the subheading of the Implementation section where a more detailed description can be found. The informatics components described in Table 1 that support patient engagement are not shown.

Our goals were to do the following:

Collect registry data directly in the EHR as part of the clinical care process, and electronically transfer them to the registry, which would eliminate the need for chart abstraction and double data entry, resulting in a data-in-once strategy that reduced the resources required for care centers to participate.27

Deploy automated reports to support chronic care management and QI activities, increasing the value of the registry’s data to clinicians both by making it easier for them to implement an evidence-based model of chronic care delivery and by enabling care centers to manage their QI efforts more efficiently.

Demonstrate the ability to use observational registry data for CER by providing researchers with a larger, more representative patient population (described in 23).

Test the feasibility of a federated registry architecture that would allow centers to retain Personal Health Information (PHI) locally, minimizing privacy and security concerns, while also providing the ability to quickly identify patient cohorts.

Pilot informatics tools to increase patient engagement by providing patients and families with a snapshot of their health status while also allowing them to communicate with their care team between visits, increasing the ability of patients to take more ownership of their own care.

Table 1 lists a selection of the most important functional requirements of the network-based Learning Health System, along with the components of the architecture that satisfy each functional requirement. Included in the component name column is the letter of the Implementation subsection where a more detailed description can be found. The process for creating research-grade data sets from observational registry data is not described here due to space constraints, and the deployment of the tools used to increase patient engagement is covered elsewhere.28

Table 1.

Functional Requirements and Corresponding Architectural Components of the Network-Based Learning Health System

FUNCTIONAL REQUIREMENT	ARCHITECTURE COMPONENT
DATA CAPTURE
Clinical documentation functionality (discretely capture data at the point of care)	EHR-based data collection forms—see Implementation section, subsection (C) below
Electronic transfer of clinical documentation responses to the registry	Electronic data transfer (see D)
Form responses added to a progress note or referral letter	EHR-based data collection forms with links to note templates (see C)
Ability for all centers to participate, regardless of EHR maturity; ability to capture non-EHR data	Web-based data collection forms (see B)
Capture of all common data elements for the condition of interest	Process for defining outcome measures and data elements necessary for computation; process for calculating derived variables (see E)
MANAGING CLINICAL CARE AND QUALITY IMPROVEMENT (QI)
QI & chronic care management reports with daily refresh	Automated reporting—performance measurement for quality improvement, pre-visit planning, and population management (see F)
Reports to monitor data entry compliance	Automated reporting—data quality and exception reports (see F)
USE OF DATA FOR RESEARCH
Ability to use the data to support clinical care, QI, and research	Standardized IRB protocols, Data Use and Business Associate Agreements (see A)
Ability to distinguish between QI and research participants	Consent management (see G)
Populating of certain analytical reports with patient identifiers	Consent management and automated reporting (see F, G)
Cohort discovery and preparation for research	i2b2 workbench (see H)
Support of observational, patient-level research studies	Procedures for creating research-grade data sets from observational registry data
PARTICIPATION OF PATIENTS
Tools to increase patient engagement	Ability to send surveys to patients in between visits, ability to provide registry data back to patients

Note: The italicized letters in parentheses indicate the subsections A–H in the Implementation Details section below where more detailed descriptions can be found.

These requirements represent a higher-level abstraction of user needs that were identified through a strategic planning process undertaken by the leadership of ImproveCareNow and also elicited through e-mails, conference calls, and webinars with numerous clinicians within the network. Development occurred in an iterative process, with volunteers recruited from participating care centers to provide feedback, perform user acceptance testing, and make recommendations for additional features. Following the IHI Breakthrough Series model for improvement,29 ImproveCareNow holds networkwide in-person Learning Sessions twice a year, along with monthly webinars. Prior to deployment, major informatics developments were presented at either a Learning Session or a monthly webinar. Upon release, they were further discussed in targeted webinars or training sessions. As the informatics platform has matured, clinicians and coordinators within ImproveCareNow have been recruited to give presentations to their peers on how they have been able to successfully incorporate the tools into practice. All of the enhanced registry tools are accessible from the same front end screen, which is shown in Figure 2.

Figure 2.

Landing Page for the Registry

Note: Users can access a variety of tools from this screen.

Implementation Details

The initial project to transition ImproveCareNow’s registry spanned three years, from September 2010 to August 2013. ImproveCareNow received additional funding from AHRQ for an 18-month grant extension, which aimed to expand the network’s capacity to transfer data electronically and to identify the research priorities of network stakeholders (e.g., clinicians, patients, and families). This extension started in September 2013 and ended in March 2015. This period also saw continued growth of the network.

A. Institutional Review Board (IRB) Protocol and Legal Agreements

Prior to this project, ImproveCareNow had relied on each center’s Institutional Review Board (IRB) to determine how to interpret the project protocol. Like other multicenter projects that involve both research and QI,30 the protocol had received a range of interpretations regarding the need for IRB oversight and patient consent. For the 27 care centers that were in the network in September 2010, these interpretations are listed in Table 2.

Table 2.

Local IRB Determinations of the ImproveCareNow Protocol for the 27 Network Centers at Project Initiation, 2010

IRB DETERMINATION	NUMBER OF CENTERS
Research protocol describes the collection and sharing of identifiable patient-level data for both research and QI purposes. Individual patients are included with a signed consent document and authorization.	19
Research protocol describes the collection and sharing of identifiable patient-level data for both research and QI purposes. Individual patients are included with a local-IRB approved waiver of the requirement to obtain a signed consent document and authorization.	2
The proposed activity represents a QI activity that does not meet the regulatory definition of research involving human subjects and, therefore, is not an activity that requires IRB review and approval or informed consent.	4
Center is pending IRB review.	2

Among the centers obtaining consent, there was substantial variation in the consent language about where the data were being sent and whether they would be shared with others. In addition, the use of Data Use Agreements (DUA) and Business Associate Agreements (BAA), which provide regulatory safeguards for the transfer of data for research and nonresearch purposes, was not routine. As a result, ImproveCareNow used the registry transition as an opportunity to reduce the variation in IRB interpretations and to standardize consent language around the use of data for research.

The network sought to establish a “triple use” registry protocol that clearly stated that data would be used for chronic care management, QI, and research. It also adopted a model where data on the entire IBD patient population could be included for care management and QI purposes—but only those patients who consented would have their data used for human subjects research purposes (dates of service, which are elements of PHI, are often used in analyses to determine the effect of a particular QI intervention across the network). For centers that had patients who previously consented under the old protocol, a guidance document was created that would allow those centers’ IRBs to determine whether existing data and consents could be grandfathered into the new protocol. ImproveCareNow also implemented a federated IRB model,31 in which Cincinnati Children Hospital Medical Center (CCHMC) would serve as the study’s IRB of record. Centers could choose to rely on CCHMC’s IRB, saving time and resources when it came to submitting amendments and continuing reviews. Finally, the network implemented standardized DUAs and BAAs to cover the transfer of patient data (see, for examples32). Approximately 45 percent of the network has elected to use CCHMC as their IRB of record. This percentage has held steady even as the number of centers in the network continues to increase.

B. Web-Based Data Collection

While one of the primary goals of the project was to enable a data-in-once architecture that leveraged the EHR for primary data collection, not all centers utilized an EHR or had an EHR that allowed for the collection of custom data elements. Some centers were in the process of transitioning from one vendor to another, and the network often seeks to collect data for smaller-scale research projects that include elements captured outside of a routine clinic visit. As a result, we needed to continue to provide the network with the ability to enter data manually through Web forms. Our solution was to utilize a forms engine that had been previously developed at CCHMC, the open-source Informatics for Integrating Biology and the Bedside (i2b2) Forms Cell.33 The data model that underlies the forms is a customized version of the Clinical Data Interchange Standards Consortium (CDISC) Operational Data Model (ODM),34 which provides a structure to represent and allow for the exchange of clinical research data and metadata. The data elements of the ImproveCareNow registry are largely coded to custom terminologies, though efforts are underway to map data from standard domains (e.g., medications and laboratory results) to terminologies like Logical Observation Identifiers Names and Codes (LOINC) and RxNorm.

C. EHR Data Collection Forms

Using an approach promoted by James and others,35 ImproveCareNow had previously developed a streamlined method to create a parsimonious set of measures covering both the processes and outcomes of care. The network had used QI techniques to help care centers embed the capture of discrete IBD-specific data elements in routine clinical workflows.25 These variables, along with patient demographics, constitute roughly 50 percent of the total possible ImproveCareNow registry variables, with laboratory results and medication data constituting the rest.

Our goal was to create standardized data collection forms that allowed the IBD-specific data elements to be captured directly in the EHR via routine clinical documentation during the patient visit. Form responses would be extracted from each center’s EHR and electronically transferred to the registry. To meet clinicians’ needs, there was also a requirement that it be possible to pull the same form responses into a progress note or referral letter, saving additional documentation time.

Prior to this effort, centers used a variety of ways to capture data in the EHR. Some centers captured the data in a text note, selecting responses from a dropdown menu. This ensured that data were available in the progress note, but required double data entry to transfer the information to the registry. Others created their own data collection tools that allowed for the discrete capture of the registry data in the EHR and for the responses to be pulled into a note, but the form build was specific to that center’s EHR, limiting scalability and reusability. Our intent was to create a single form for each vendor that could be installed by all of that vendor’s customers.

We decided to focus on the vendors that comprised the majority of the network’s EHR installations. Epic, Cerner, and General Electric represented the vast majority (∼75 percent) of the network’s EHR install base at the time (late 2010, early 2011). We approached each vendor and requested its assistance in creating IBD-specific EHR form templates. We found significant variation among them in terms of the time they required to complete the build, their overall interest in supporting this type of development, and whether or not they charged for their services. We had the most success with Epic, who assigned a small team to the project, developed the form free of charge, and then made it available to all their customers through their normal deployment channels.

D. Electronic Data Transfer

EHR data are uploaded to the registry as a flat file, comma-separated values (CSV) format, either manually through the registry portal or via an automated process. Once a file has been uploaded, it is scanned and checked for errors, such as the presence of an unknown participant or value, or missing identifiers. Users can then take steps to resolve the issues (e.g., register participant, skip offending record). Records that pass quality control have their EHR-based identifiers replaced with registry identifiers, and the resulting data are stored using Web services of the Forms Cell.

E. Process for Calculating Derived Data

The outcome and process measures that are used to support the network’s clinical and QI activities are calculated using a mix of raw registry data and derived variables. Both the derived variables and measures are generated on a nightly basis using a set of automated SAS procedures that execute against the registry database, which runs on Microsoft SQL Server. In its current form, this process creates a series of data sets that serve as the basis for the network’s automated reports.

ImproveCareNow computes a series of 10 outcome measures, 7 process measures, and 8 data quality measures. Examples of each type of measure are displayed in Table 3 below. The measures and their operation definitions are documented in a wiki that can be accessed by network participants, though there is an effort underway to make this information publicly available.

Table 3.

Example Outcome, Process, and Data Quality Measures Used by the ImproveCareNow Network

MEASURE	DESCRIPTION
OUTCOME MEASURES
Percent of patients in remission	Numerator: Total number of patients with physician global assessment (PGA) identified as “quiescent”Denominator: Total number of patients who were seen in the last 13 months
Percent of patients in growth failure	Numerator: Total number of patients with growth status identified as “In Failure” at their most recent visit.Denominator: Total number of patients seen in the last 13 months
PROCESS MEASURES
Percent of ICN visits with a complete bundle	Numerator: Number of visits with documentation of all bundle components: Disease phenotype (Crohn’s disease only) Extent of disease Severity of disease (PGA) Height, Weight, and body mass index (BMI) plotted Nutrition and growth status classified Diagnosis Denominator: Total number of visits during the report month
Percent of visits where initial dose of anti-Tumor Necrosis Factor (TNF) therapy is given and patient had a tuberculosis (TB) test within the prior 12 months	Numerator: Total number of visits where patient has been tested for TB within 12 months (365 days) prior to the induction dose date of anti-TNF therapy.Denominator: Total number of visits where the first induction dose of anti-TNF therapy has been recorded
DATA QUALITY MEASURES
Percent of visits with all critical data present	Numerator: Number of visits with all critical variables presentDenominator: All visits
Percent of hospitalizations entered within 30 days of discharge	Numerator: Number of hospitalizations where the entry date is no more than 30 days apart from the discharge dateDenominator: Total number of hospitalizations with a nonmissing discharge date

F. Automated Reports

The network employs a number of reports to support its chronic care management, QI, and research activities. These reports are described in Table 4. They are generated using Microsoft SQL Server Reporting Services (SSRS), an enterprise-level reporting tool. The look and feel of the QI and data quality reports reflect past experience and best practices in process improvement. The content and format of the care management reports (population management and pre-visit planning) were modeled on existing paper and Microsoft Excel-based reports. Examples of the reports are shown in Figures 3 and 4 below.

Table 4.

Description of the Automated Reports Accessible Via the ImproveCareNow Enhanced Registry

TYPE OF REPORT	DESCRIPTION
Population Management	Helps clinicians identify subpopulations. The report provides aggregate information on each center’s patient population according to metrics like demographics, medication usage, clinical status, and risk assessment. It also allows users to drill down into each metric and view patient-level data on all patients with matching criteria.
Pre-visit planning	Used to help plan upcoming clinic visits, these reports provide a snapshot of the patient’s current status and ensure patients are receiving proper medication dosing. They include information on diagnosis and disease phenotype; selected information from past visits; a patient’s current risk assessment; and considerations (recommendations) for medication dosing, lab ordering, and other actions based on the severity of disease.
Monthly quality improvement (QI) measures	Provide information on both the performance of individual centers and the network as a whole on a variety of process and outcome measures. They are delivered in the form of run charts, control charts, and dashboards (including small multiples). Example measures include remission rate, whether a medication dosage adheres to standard guidelines, and whether the visit documentation is complete.
Data quality reports	Similar to the QI reports, the data quality reports provide information on the quality and completeness of the data that are being entered into the registry. They are also delivered to centers as run charts, control charts and dashboards.
Exception reports	List the patients and/or visits that fail the data quality reports, and—in some cases—the variable that is causing the failure.

Figure 3.

Example QI and Data Quality Reports

Note: Metrics can be viewed via a dashboard (a), graphs of small multiples (b), and as control charts (c).

Figure 4.

Initial Version of the Pre-Visit Planning Report

Note: Diagnosis and phenotype data are provided in the top section, followed with selected values from previous visits. Lab results that are out-of-date based on the patient’s treatment regimen are labeled in pink. The patient’s risk stratification score is listed in the next section, followed by information on any medications or treatments they are taking. If a medication was not dosed according to the specified guideline, a note would be listed in the “Attention Needed” column (bottom right).

Soon after deploying the care management reports, we received feedback from users about usability issues, such as cumbersome workflows when trying to access multiple reports (many centers do pre-visit planning once a week and generate reports for all patients scheduled in the next week) and formatting issues when trying to print them. Working with a group of power users, along with faculty and graduate students with experience in interaction design and information visualization, we initiated a redesign of the care management reports. We interviewed participants to determine how they interacted with the reports as well as the information they needed to see in each section. This led to the creation of several design prototypes that were shared with the network. After gathering feedback, the most popular prototypes were translated into production-level reports. These new reports, shown in Figures 5 and 6, were deployed along with new methods of access that greatly enhanced user efficiency.

Figure 5.

Redesigned Pre-Visit Planning Report

Notes: Diagnosis and phenotype are highlighted in yellow and symbols have been added to indicate where attention is needed. Treatments are only listed if a patient is taking them, saving space when printing.

Figure 6.

Longitudinal Version of the Pre-Visit Planning Reports

Note: Clinicians can view multiple measures of a patient’s status over time, as well as previous treatments. This information can be helpful when determining a new treatment plan if a patient is not responding to the current treatment.

G. Consent Management

At the start of the project, the ImproveCareNow registry consisted of only those data elements that a limited data set comprises. The clinicians in the network requested that the pre-visit planning and population management reports be populated with the names and medical record numbers of their patients, which would increase their clinical utility. As a result, a separate Web application, called “consent management,” was developed to store PHI and track patient consent. This application is integrated with the rest of the registry architecture, and is seamlessly accessible from the same front end as the rest of the tools. To generate the identified reports, all registry-based identifiers are replaced with actual patient identifiers at runtime.

The consent management application stores demographic information on participants, tracks their consent status, and can send alerts to the study staff when a participant’s consent expires (e.g., pediatric centers often see patients with chronic conditions into early adulthood. Once a patient turns 18, they legally become an adult and a new consent is needed in order to continue collecting data). It also includes a web-based e-consent module, which is currently being tested as a way to lower the transaction costs of obtaining informed consent.

H. Cohort Identification

One of the key elements in positioning ImproveCareNow as a Learning Health System was to increase the network’s capacity to engage in research, particularly as it relates to clinical trials. To support these endeavors, we integrated cohort identification tools with the registry database, allowing users to quickly generate hypotheses and determine study feasibility. The registry supports two levels of cohort identification. Using a custom version of the i2b2 workbench36,37 (Figure 7), care centers can run queries against their own patient population, generating aggregate results with the ability to drill down into patient level data. We have also created a virtual Shared Health Research Information Network (SHRINE) among the i2b2 databases hosted at the data coordinating center,36–39 which allows authorized users to generate aggregate numbers for the network as a whole through a custom version of the SHRINE workbench, which is very similar to the i2b2 workbench, but lacks the ability to drill down into patient-level data.

Figure 7.

Example i2b2 Interface Used for Cohort Identification

Note: This query is intended to find all males with a current diagnosis of Crohn’s disease who have ever been on a biologic.

Results

At the start of the original grant in September 2010, there were 27 centers in ImproveCareNow. By the end of the initial project period in August 2013, the network had grown to 53 care centers. During the 18-month grant extension, ImproveCareNow grew by another 20 centers, with 73 centers participating in the network as of March 2015.

The Epic IBD SmartForm and the associated process to extract the responses were made available to all Epic customers in mid-2011. At the end of the initial project in August 2013, 9 centers were using the Epic SmartForm. By the end of the 18-month grant extension, that number had risen to 31. SmartForm data can be collected in about two minutes during routine care for follow-up patients. Cerner built an IBD PowerForm, and made it and the extract process available to the network in February 2014, though centers requested modifications that were not completed until late 2014. The form is now in production at one Cerner center, and is being implemented at several more. The form for Centricity is currently in testing at one center. As the network has grown, Allscripts has become the third largest vendor in terms of install base (as of March 2015, the breakdown for the 73 centers in the network is 57 percent Epic, 15 percent Cerner, 7 percent Allscripts, 4 percent Centricity, 4 percent eClinicalWorks, and 13 percent “Other”). We have collaborated with Allscripts and several of the Allscripts ImproveCareNow centers on the creation of a form, but progress is slow.

The number of centers that are transferring EHR data to the registry is lower than the number that are collecting data in the EHR because of a lag between the form implementation and the extraction process implementation, but we are seeing steady increases. At the end of the initial project in August 2013, only 3 centers were transferring data to the registry. By the end of the grant extension 18 months later, we had seen a seven-fold increase, to 21 centers. Figure 8 represents the progress that the network has made in enabling the transfer of EHR data.

Figure 8.

Number of Patients Whose EHR Data Have Been Uploaded to the Registry

Notes: The “Goal” line represents the target set as part of the 18-month grant extension, which was 75 percent of ImproveCareNow’s patient population at the time of submission. The “Number Patients” line represents the number of patients whose EHR data have been uploaded.

One of the aims of the grant extension was to increase the transfer of EHR data to 75 percent of the patients in the network at the time the grant extension was submitted. That number (7,682) is represented by the “Goal” line of the figure. The number of patients that have had their data uploaded is indicated by the “Number Patients” line. As shown in the figure, the network surpassed its target in early 2015.

Centers that have enabled the data transfer process report a significant time saving over manual data entry, of approximately 7 minutes per patient visit.

The average ImproveCareNow center has about 300 patients, which—assuming 3 visits/patient/year—means that each center saves approximately 100 hours a year in time spent on data entry. That corresponds to 2 hours each week—5 percent of a full-time employee (FTE)—that can be dedicated to other tasks. We are currently extending the electronic data transfer process to eliminate the manual entry of lab results and medication orders, which we expect will save an additional 7 minutes per patient visit, or another 5 percent FTE per center per year.

The ability to quickly identify cohorts of interest within the registry using i2b2 has allowed ImproveCareNow to respond to several inquiries from pharmaceutical companies interested in clinical trials and has led to at least one signed contract for a descriptive analysis of a subset of the registry. Additional discussions are ongoing for more substantial engagement. Individual investigators within ImproveCareNow have also used the tools to determine whether enough patients exist for a potential study. We are developing plans on how to provide appropriate training and support, which would allow these tools to be deployed more broadly across ImproveCareNow.

During the implementation of this architecture, ImproveCareNow has seen continued improvements in the remission rate. Figure 9 shows the remission rate for centers that have been in the network long enough to have entered at least 75 percent of their patient population into the registry. At the start of the project, the remission rate for these centers was approximately 73 percent. At the end of the grant extension, the remission rate was above 78 percent. While it is not currently possible to attribute this increase directly to the implementation of the registry or the associated tools, we believe it has played a factor, since we have seen increases in other important measures such as the quality and completeness of data. Regardless, this increase represents hundreds of additional children and adolescents whose disease is now in remission.

Figure 9.

Percentage of Patients in Remission in Any Given Reporting Month (Centers with >=75% of Their Population Enrolled in the Registry)

Discussion

The new data-in-once registry architecture provides a number of benefits to the ImproveCareNow Network compared to their previous system. Two observed benefits include a significant reduction in the time dedicated to data entry and the increased availability of chronic care management reports. The latter provides a direct benefit to care centers for the time they have spent on registry data collection. And the more time that centers invest in ensuring that they enter accurate and complete data, the more beneficial the reports become. This architecture serves as a proof of concept of what is possible in a Learning Health System. While it provides demonstrable benefits, our experience has shown that there are still a number of challenges that must be resolved before it is possible to create similar systems at a national scale.

We were able to successfully develop EHR data collection forms for three different vendors, but one of the drawbacks to the current process is that a custom form must be created for each one. A more scalable approach would be to leverage a model like that one proposed by the Structured Data Capture (SDC) initiative.40 In this model, a single form is defined once and placed into an external library. It can then be implemented in any EHR that supports the SDC standard. As long as the form responses can be stored in the EHR and pulled into a progress note, the lack of which which was a shortcoming of the predecessor Retrieve Form for Data Capture (RFD) standard,41 SDC would be a viable alternative to vendor-specific forms. For the time being, however, networks must work with multiple vendors. An approach to speed the form development process is to have one of the participating centers create the relevant forms and provide them back to the vendor to package for deployment. This requires centers to agree on content and workflow (most EHRs provide several different ways to collect the same data), but may help expedite the process.

Although there is great enthusiasm for the EHR forms among ImproveCareNow clinicians, it has taken a fair amount of time to implement them locally. In surveying the network, we uncovered significant barriers to deployment, including the following:

The need to engage institutional leadership to drive the transition because of competing organizational IT priorities (e.g., ICD-1042 and Meaningful Use1);

The difficulty for some institutions to devote even modest resources (about 20 hours) to implement the forms;

The isolation of ImproveCareNow center physician leaders from hospital IT;

Delays caused by system upgrades or the transition to a new EHR.

We believe that these barriers are likely to be temporary, but can be expected to last through the end of the decade as the health care industry grapples with the later stages of Meaningful Use and similar regulations.

To transfer data to the registry, we use the relatively straightforward method of flat file uploads. There are more sophisticated methods available to exchange EHR data, such as HL7 interfaces or Web services, but it was not feasible to use these mechanisms to transmit the form responses. We found that even the creation of a simple flat file often proved to be a hurdle for some care centers’ IT departments. We estimate that the effort to set up the process to generate the file requires roughly 20 hours of effort on the part of the center. In many cases, largely because of competing institutional priorities, even this was considered to be too much of an outlay. Even offering to compensate centers for their time proved to be no help. Due to this experience, which was repeated across numerous care centers, we have largely avoided the exploration of more complicated extraction methods because of our concerns about implementing them at scale.

One possible mechanism for ImproveCareNow to obtain more complete EHR data on its patients is to work through a distributed research network (DRN), like the National Patient-Centered Clinical Research Network (PCORnet). Institutions that participate in these networks have staff that are focused on the creation of standardized extracts of EHR data for their entire patient population. ImproveCareNow could work with the staff at each institution (or network coordinating center) to obtain a data extract for those patients that are followed at that institution’s IBD clinic. This method is appealing, but presents several challenges. DRNs operate on a much slower time scale than do improvement networks when it comes to the timeliness of their data. Data in a DRN are often refreshed on a quarterly or semiannual basis, while improvement networks need data in as real-time a fashion as possible. Also, depending on the IRB protocols and legal agreements that govern the DRN and improvement network, using a DRN as a data source may require negotiating additional agreements or protocols. Finally, while most DRNs receive a significant amount of startup funding, sustainability is always a concern (improvement networks face similar issues).43 IT staff at ImproveCareNow care centers are not paid to create the EHR extracts that are uploaded to the registry. It is unlikely that a DRN would be able to provide data to an outside network free of charge, and it is unknown whether improvement networks could absorb this additional cost. Even so, this approach remains one of the most promising ways for improvement networks to obtain large amounts of EHR data on patients for research purposes. There remains a need to obtain the EHR data that are required for care management and improvement as quickly as possible, however.

The architecture supporting the ImproveCareNow enhanced registry currently operates in a centralized fashion, with data submitted to a coordinating center. We explored the feasibility of creating a distributed registry using the SHRINE federated query platform to exchange data. By default, SHRINE only allows the exchange of aggregate results, though in previous work we extended SHRINE to allow the transfer of patient-level details.44 We found that several issues prevented us from being able to deploy the ImproveCareNow registry in a truly geographically distributed fashion:

Delays in setting up the registry because of network and firewall issues;

The ongoing cost to centers of funding the required personnel to set up and maintain the distributed infrastructure;

Difficulties packaging the procedures needed to compute the derived variables and outcome measures so they could be executed outside the coordinating center without user intervention;

Difficulty in maintaining and modifying those packages as the needs of the network evolves;

The inability to calculate metrics using SHRINE that were not based on patient counts; and

The cost of developing and maintaining the interface between the i2b2 and SHRINE Web services and enterprise reporting tools like those used by ImproveCareNow to produce their automated reports.

Distributed research tools are not designed to support the kinds of on-demand reporting needed to support the QI and clinical care activities of the network. To be feasible, these tools would need to be adapted to support ImproveCareNow’s data management and reporting functions

Suggestions for Future Use and Implementation

We have identified a number of enhancements that we intend to add to our registry architecture in the future. These include creating interfaces to send and receive data from external applications, providing tools for data discovery, and mapping registry elements to standard terminologies. In the long run, we see the future of the registry as being a platform for decision support. This includes the ability to return measure definitions and results to the EHR, as well as the ability to send information like pre-visit planning considerations. The emerging Quality Document Reporting Architecture (QDRA) and Health eDecisions (HeD)45 standards may enable such workflows.

The desire to position the registry as a platform for decision support is two-fold. The first is that the natural home for most of the analytical and reporting tools provided by the ImproveCareNow registry is the EHR, where clinicians can review information and quickly order changes in therapy. The second is that it will simply not be possible for each EHR vendor to develop the specific measures, workflows, decision support, or content for every condition or rare disease. It will be necessary to adopt a crowdsourced approach that harnesses the capacity and expertise of the participants. Networks provide the community, expertise, and motivation to do this. But to be feasible, there must be standardized approaches to allow forms, decision support, and reports to be deployed across vendors.

Conclusion

In a Learning Health System, clinical care, QI, and research operate as integrated activities. The architectural design to support all three of these activities is very different from an architecture that supports just one, or even two, of them. At the current time, there is a lack of existing standards that would allow for the implementation of a Learning Health System at scale, though several, such as the SDC and HeD, may be a step in the right direction. We have shown the potential and the possibility of a network-based Learning Health System. Our hope is that this provides a roadmap for others interested in following a similar path and serves as a call for policymakers and sponsors to accelerate the process of defining the necessary standards and methods as well as the best practices on how to adopt them.