An initiative using informatics to facilitate clinical research planning and recruitment in the VA health care system.

BACKGROUND: Randomized clinical trials are the gold standard for evaluating healthcare interventions and, more generally, add to the medical knowledge related to the treatment, diagnosis and prevention of diseases and conditions. Recent literature continues to identify health informatics methods that can help improve study efficiency throughout the life cycle of a clinical trial. Electronic medical record (EMR) data provides a mechanism to facilitate clinical trial research during the study planning and execution phases, and ultimately, can be utilized to enhance recruitment. The Department of Veterans Affairs (VA) has a strong history of clinical and epidemiological research with over four decades of data collected from Veterans it has served nationwide. The VA Informatics and Computing Infrastructure (VINCI) provides VA research investigators with a nationwide view of high-value VA patient data. Within VA, the Cooperative Studies Program (CSP) Network of Dedicated Enrollment Sites (NODES) is a consortium of nine sites that are part of an embedded clinical research infrastructure intended to provide systematic site-level solutions to issues that arise during the conduct of VA CSP clinical research. This paper describes the collaboration initiated by the Salt Lake City (SLC) node site to bring informatics and clinical trials together to enhance study planning and recruitment within the VA.
METHODS: The SLC VA Medical Center physically houses both VINCI and a node site and the co-location of these two groups prompted a natural collaboration on both a local and national level. One of the functions of the SLC NODES is to enhance recruitment and promote the success of CSP projects. VINCI supports these efforts by providing VA researchers access to potential population pools. VINCI can provide 1) feasibility data during study planning, and 2) active patient lists during recruitment. The process for CSP study teams to utilize these services involves regulatory documentation, development of queries, revisions to the initial data request, and ongoing communications with several key study personnel including the requesting research team, study statisticians, and VINCI data managers.
RESULTS: The early efforts of SLC NODES and VINCI aimed to provide patient lists exclusively to the SLC CSP study teams for the following purposes: 1) increasing recruitment for trials that were struggling to meet their respective enrollment goals, and 2) decreasing the time required by study coordinators to complete chart review activities. This effort was expanded to include multiple CSP sites and studies. To date, SLC NODES has facilitated the delivery of these VINCI services to nine active CSP studies.
CONCLUSION: The ability of clinical trial study teams to successfully plan and execute their respective trials is contingent upon their proficiency in obtaining data that will help them efficiently and effectively recruit and enroll eligible participants. This collaboration demonstrates that the utilization of a model that partners two distinct entities, with similar objectives, was effective in the provision of feasibility and patient lists to clinical trial study teams and facilitation of clinical trial research within a large, integrated healthcare system.

Introduction

Randomized clinical trials are the gold standard for evaluating healthcare interventions and, more generally, add to the medical knowledge related to the treatment, diagnosis and prevention of diseases and conditions [[1], [2], [3], [4], [5], [6]]. Recent literature continues to identify health informatics methods that can help improve study efficiency throughout the life cycle of a clinical trial [[7], [8], [9], [10]]. Electronic medical record (EMR) data provides a mechanism to facilitate clinical trials during the study planning and execution phases, and ultimately, can be utilized to enhance recruitment [5,9]. EMR data used to help inform the study and protocol development process is often referred to in context of feasibility [11]. This type of data is used to assess internal and environmental capacity, and the alignment of a clinical trial with the environment in which it is conducted, in terms of its study design, dose of investigational product, comparator, and patient type [12]. Addressing these questions early may conserve resources and address barriers to recruitment [5,12,13]. Furthermore, feasibility data is used as a source of information to enhance trial recruitment [5,12,14,15]. A major challenge for researchers is the identification of eligible subjects in a timely manner [16]. EMR data can be employed as a feasibility tool for study teams due to its capacity to be truncated into concise, focused potential study participant lists that can be employed to expedite the participant screening process [5,6,12,13].

The United States Department of Veterans Affairs (VA) has a strong history of clinical and epidemiological research with over four decades of data collected nationwide from Veterans it has served [17]. The VA boasts the largest integrated health system in the United States and provides comprehensive care (inpatient, outpatient, mental health, rehabilitation, pharmacy, long-term care services, etc.) to more than nine million enrolled Veterans each year [18]. These data are acquired using an interface called the Computerized Patient Record System (CPRS) and capture medical information spanning across 144 VA Medical Centers (VAMCs) and 1221 outpatient clinics [19]. CPRS is used for documenting all clinical activities, including but not limited to, assessments, diagnoses, interventions, laboratory results, imaging, medication orders, consultations, etc. This EMR data is relatively standardized across VAMCs and outpatient clinics and stored in a repository called the Corporate Data Warehouse (CDW). The CDW is considered one of the most remarkable and robust data warehouses undertaken by a health care system because of its capacity for the storage of massive amounts of data and for its provision of advanced data management and analysis software [20]. Although the VA's primary purpose for EMR data was initially to support and track health care delivery performance, the CDW has evolved to become a valuable resource for researchers [20].

The VA Informatics and Computing Infrastructure (VINCI) is a Health Services Research & Development (HSR&D) Resource Center that provides investigators with a nationwide view of high-value VA patient data. VINCI includes more than 20,000 analysts, program managers, investigators, and research entities that specialize in providing data services for research. Retrieving specific data can be a challenging task for researchers to navigate due to the vast amount of data available and variability in how data is retrieved within the CDW. VINCI is of interest to researchers because of their capability to assist researchers to access and analyze the CDW data.

VINCI provides an extensive range of research capabilities for a wide variety of clinical scientists interested in capitalizing on VA's EMR. Among these, includes a partnership established with the VA Cooperative Studies Program (CSP), a national clinical research infrastructure embedded within VA's integrated healthcare system [21]. The VA CSP established the Network of Dedicated Enrollment Sites (NODES) as a consortium of nine sites to generate and implement systematic site-level solutions to more efficiently recruit for CSP studies [22,23]. The Salt Lake City (SLC) VAMC physically houses both VINCI and a node site and the proximity of these two groups prompted a natural collaboration on both a local and national level.

This manuscript describes the early experiences of the collaborative efforts initiated by the SLC NODES to establish a partnership that brings informatics and clinical trials together to enhance study planning and conduct within the VA.

Methods

One of the goals of the SLC NODES is to enhance study recruitment and promote the success of CSP projects. VINCI is a resource that supports these efforts by providing VA researchers access to enriched patient population pools. For CSP, these services can be most helpful during two stages of a trial: 1) the collection of feasibility data during the study planning phase, and 2) the creation of potential study subject lists during the recruitment phase.

Feasibility data allows research groups to more accurately assess potential patient population groups at VAMCs that are under consideration to serve as study sites. This data also has the potential to highlight additional sites that have adequate populations of a desired patient group and that should also be considered at the feasibility stage. Additionally, these data may shape protocol development by providing information about the stringency of certain inclusion and exclusion criteria. This can be done by conducting a feasibility funnel analysis. The prospective patient population pool can vary significantly, narrowing or broadening the total number of potential subjects with each designated criterion (Table 1). The funnel analysis will show how many potential patients are removed based on the inclusion and exclusion criteria. Each row in the analysis corresponds to the designated criterion. This is beneficial because it allows the research team to see specifically which criterion narrows the patient pool, and by how much. A best practice for feasibility work is to first determine the required sample size for a respective study, examine the impact of its inclusion/exclusion criteria, and review the types of data limitations that exist and can be resolved.

Table 1

Key Points to Include in an Attrition table.

Description	Example
1. Provision of study location parameters	Nationwide versus targeted sites or regions
2. General time-frame for the study and each diagnosis, procedure, test, prescribed medication	Ever; last five years; calendar year/fiscal year; last six months, etc.
3. Provision of time-frame frequency and units for the lab test	At least one in last six months; if inclusion/exclusion criteria or lab test out of normal range; normal range should be provided
4. Provision of diagnosis and all ICD9/10 codes	ICD9 codes if study time-frame is before September 30, 2015; ICD10 codes if study time-frame is after October 1, 2015 or both
5. Provision of medication list	All drug name (generic and brand); drug route, time, duration for prescription

A second service involves creating active VAMC patient lists that can be used for study recruitment activities. Depending on the individual needs of the research team, these lists can be very specific, identifying patients that are likely to be eligible (based on study inclusion/exclusion criteria), or more broadly, to identify groups of patients who might be appropriate for additional screening. The process of developing these lists involves operationalizing the inclusion and exclusion criteria that are to be queried and identified within CDW. For example, identifying criteria that are ambiguous, such as “mild or moderate traumatic brain injury (TBI)”, requires clinical experts to define what and how these are delineated in the medical record.

In our experience to date, this process requires multiple meetings (in person or virtual), that result in query revisions and ongoing communications with key stakeholders that must include the Study Chair, the requesting research team, study statisticians, and VINCI data managers. The meetings are efficient and often brief, though “buy-in” from all key participants are imperative for a successful outcome. SLC NODES serves as a facilitator and point of contact for all CSP study sites and works directly with VINCI data managers and operational staff to facilitate the communication process and the development and distribution of data requests. Awareness of the timeframe in which these patient lists should be made available for the study teams is important when navigating the necessary data use approval processes. Of note, a significant amount of effort is required to “translate” the initial data request into a functional and well-designed query. This is accomplished by ongoing communication between the data managers, SLC NODES and the requesting study team. Examples of such “translation” are displayed in Fig. 1. SLC NODES also assists with the security and regulatory issues inherent with data access provisions.

Fig. 1

Model for translation of initial data request into query form.

Results

The early efforts of SLC NODES and VINCI intended to provide patient lists exclusively to the SLC study teams for the following purposes: 1) increasing recruitment for trials that are struggling to meet their respective enrollment goals, and 2) decreasing the time required by study coordinators to complete medical chart review activities. These efforts were expanded to include multiple CSP sites and studies. To date, SLC NODES has facilitated the delivery of the VINCI services as described to nine CSP studies. Throughout this process, important lessons were learned. The following case examples describe these activities throughout the planning and execution phases of the referenced clinical trials.

The approaches to feasibility and patient list acquisition are very similar with regards to how the requests are processed, the difference being what the requesting party receives for each type of request. Feasibility data requests are deemed preparatory to research, therefore they only provide total aggregate numbers of potential subjects, whereas developing patient lists provides researchers with protected health information (PHI) and full access to the data.

The first multisite feasibility request that was fostered through this collaboration was for a diabetic kidney disease study during its planning phase. This request included Veterans who had received a diabetes mellitus diagnosis at any point in time, had an estimated glomerular filtration rate (eGFR) lab value between 20 and 60 in the calendar year, and had a urine albumin-to-creatinine ratio (UACR) lab value greater than 300 in the calendar year. Data were generated for all nine node sites and resulted in the identification of more than 1700 potentially eligible study participants. From the report, it was evident that two sites had inaccurate results. Seven of the nine sites had a minimum of 85 patients identified and two sites had less than fifteen. It was determined that the inaccuracy was related to lack of standardization on how the lab values were recorded within CDW. The query was corrected accordingly and a subsequent request was later generated for all VAMCs nationwide that identified 14,790 potentially eligible subjects. The data provided from these reports helped inform the CSP scientific peer review committee's recommendation to approve the study for funding.

In a similar example, a feasibility data request was initiated during the planning phase of a study investigating the effectiveness of allopurinol and febuxostat for the prevention of gout, and later evolved to include lists of potentially eligible subjects that were provided to active study sites to support recruitment efforts. Again, it was determined that the lab values for two of the 25 sites were noticeably inaccurate. The inclusion criteria provided to the data managers included patients with a history of gout who had a serum urate level greater than or equal to 6.8 mg/dl. The exclusion criteria were extensive and required several ongoing discussions with study leadership to further identify how these criteria were captured in the medical records. Subsequent efforts were made to correct the discrepancies in how the lab data were provided at the respective VAMCs. With each review, VINCI data managers more accurately distinguished patients in CDW with serum urate levels in the proposed range. We learned that due to the dynamic nature of these records, current data may not always match past data and that the processes used for future data requests may require some level of revision to address their limitations/weaknesses. These were important lessons learned for future data requests using the same lab values, as using lab results in the data mining process can be error-prone and querying specific sites about how lab results are reported is occasionally necessary to yield accurate results. VA will be addressing data standardization issues going forward as it implements the Cerner EHR, which will use the same configuration and code set for all VA sites nationally. Additionally, VINCI has been addressing the standardization issue by transforming CDW and other high-value datasets into the Observational Medical Outcomes Partnership common data model, which takes local codes and maps them to standard terminologies [24].

When compiling feasibility data for a cardiology study, it was determined that the International Classification of Diseases (ICD) and Current Procedural Terminology (CPT) codes had the potential to be problematic for this process. Data was collected for the number of patients that had a ST-elevation myocardial infarction (STEMI) diagnosis and were hospitalized at all VAMCs nationwide during the 2015 calendar year. Several sites were presumed to be underestimated in terms of the expected patient population groups. ICD and CPT codes from both 2015 and 2016 were necessary to more accurately capture all patients. Identifying pertinent codes appropriate for capturing these patients required the involvement of clinical experts that had expertise related to the specific disease or subject matter.

Additionally, identifying medication usage in the medical chart was challenging at various times. Typically, this requires that requesting research teams verify their definitions of active medications and prescription adherence (i.e. prescription filled or not filled), and the duration of those prescriptions. They must also specify all medications within certain drug classes and their respective encompassing drug names (proprietary and generic), and application type.

While a lot of time is spent on the identification of specific measures to be captured in CDW, there are instances in which certain data points in the medical chart are not transferred or must be coded on a medical center basis. Given the vast amount of lab data present in CDW, this can be a challenging task. Some research documents and patient flags (e.g. flags for research study enrollment, scanned informed consent forms, etc.) are among those that may or may not be transferable. Conversely, a simple query of patients who had a lab Clostridium difficile test, which was either positive or negative, was complicated by the data being reported in different areas of the chart. At each site the location of the result had to be determined within the medical chart (i.e. text search versus a value search). The importance of having knowledgeable personnel review drafts of the data queries cannot be over emphasized. In our experience, these are the individuals who were able to identify errors and inaccuracies, and they played a key role in remedying or clarifying any issues so that the data provided was accurate. These data requests have also provided learning lessons and experience related to the regulatory complexities of utilizing multi-site data for the purposes of study planning (feasibility) and execution (recruitment) activities. If the clinical trial is a multi-site study, it is helpful to include all possible sites during the initial request. It is also necessary to clearly specify what specific PHI is being requested in the initial data request agreement. Otherwise an amendment and review to the request is required. Determining how the patient lists are disseminated early on is beneficial as there are two main options when multiple sites are involved. The first approach is that all study sites are granted direct access to the VINCI workspace to view and download the patient lists. This method requires that the appropriate regulatory documents are collected and submitted from each site with the data request application; these must also be updated on a reoccurring basis. The second method involves the study leadership team being granted access to receive the patient lists which in turn also necessitates their acceptance of taking the responsibility to securely disseminate the lists to all study sites. This latter strategy requires less regulatory documentation, and is generally the preferred approach of the research study leadership teams.

SLC NODES and VINCI are actively involved in facilitating the distribution of patient lists for multiple CSP studies and providing studies in planning feasibility data as requested. Both groups continuously look for opportunities to streamline the process and improve the efficiency of providing active patient lists to study sites.

Discussion

The ability of clinical trial study teams to successfully plan and execute their respective trials is contingent upon their proficiency in obtaining data that will help them recruit and enroll eligible participants in an efficient and effective manner. This collaboration demonstrates that the utilization of a model that partners two distinct entities, with similar objectives, can be effective in the delivery of feasibility and patient lists to clinical trial study teams and facilitation of clinical trial research within a large, integrated healthcare system.

To date, there is a substantial amount of published literature on the utilization of EMRs to identify potentially eligible study participants and facilitate clinical trial enrollment [[25], [26], [27], [28], [29]]. There is a limited amount of published information on the development and implementation of an archetype that allows for a continuous and iterative model that involves health informatics, clinical, project management, and study team groups in the creation of tools (feasibility data and potential participant lists) to enhance trial recruitment in a single healthcare system as large as the VA, or uses data from a repository as robust as the CDW. The only other identifiable effort of this magnitude is the EHR4CR project which aims to demonstrate how data held in EMRs can be used to enhance clinical research processes, in a multi-national context, while providing protocol feasibility, patient identification and recruitment, and clinical trial conduct and serious adverse event reporting services [30,31].

There are potential limitations to this initiative that may influence the generalizability of our experiences to other groups, as well as the utility of this approach in other settings. First, this strategy was applied in a large, integrated healthcare system that employs an electronic medical record for the collection and storage of patient health information. Although the majority of hospitals have either implemented EMRs partially or completely (58.6%), there are some that have been reluctant to utilize them [32]. This is largely because there are several financial considerations, including adoption, implementation, and maintenance costs, along with loss of revenue associated with temporary loss of productivity during these phases that discourage medical facilities to adopt and implement an EMR [33,34]. Furthermore, EMR data is primarily generated for clinical activities e.g. assessments, diagnoses, treatment, etc. and not for research purposes. Even though EMR data in the VA is relatively standardized across VAMCs and outpatient clinics, the nature of its primary purpose does not necessarily make it easily accessible for research activities and requires a dedicated infrastructure to not only navigate through the vast amount of data that is available, but to deal with the variability that exists with regards to specific types of EMR data. Successfully modeling this approach in a medical facility or healthcare system without having a core informatics group in place like VINCI would require the development of an EMR that is completely standardized and/or designed with the intent that it would be utilized for research purposes so that health services researchers at those institutions would not be presented with some of the challenges that are encountered by those in the VA health services research community. In our efforts to overcome some of these challenges, VINCI has aimed to increase the accessibility and reusability of the queries performed in the VA EMR among different VA sites and investigators by building all queries to use the entire code set to ensure that they work at any site for any time period in VA, including for Veterans who move between sites or use more than one site at a time. We have also translated the queries into standard algorithms that can be used outside of VA and these are being released through the Phenotype KnowledgeBase and the Observational Health Data Sciences and Informatics ATLAS cohort definition library [35,36]. Considering these limitations, this initiative demonstrates several key strengths. The sheer magnitude of clinical data that the CDW houses and the vast number of Veterans that the VA provides care for makes it an ideal setting to conduct clinical research in. The partnership that the SLC NODES and VINCI has demonstrated provides the capability to not only assist with cohort development, study planning, and implementation activities for the CSP and other investigators and research entities within the VA, but also has the potential to provide this type of service to research entities external to the VA e.g. other US government federal agencies, the pharmaceutical industry, etc. VA research works through partnerships and committees to share best practices, methods, and tools. A priority for VA is increasing Veterans' access to high-quality clinical trials. Efforts to achieve this priority have led to the creation of a stakeholder group consisting of federal agencies, non-profit corporations, patient advocacy groups, contract research organizations, pharmaceutical companies, and other organizations to establish more efficient processes and capabilities within the VA health care system to enable more efficient start-up of multi-site clinical trials, and to increase the number of Veterans participating in federal and industry sponsored clinical trials.

Another key strength of this partnership is that both NODES and VINCI have the duality of being components of the study sponsor (CSP and HSRD), respectively, while also providing expertise in distinct areas that advance how the research of those sponsors is conducted. NODES is a site-based consortium that can provide site-level insight (VA medical facilities) to CSP study coordinating centers, study teams, and VA research leadership on issues that arise during the execution of trials, as well as develop strategies to address them. VINCI provides VA research investigators with a nationwide view of and access to high-value VA patient data for trial planning and execution activities. These relationships promote a continuous process improvement model that allows for the development, refinement, and implementation of study feasibility and execution processes that involves investigators, study sponsors, and research teams in a unique manner.

Conclusions

In summary, this partnership has led to the development and implementation of processes that have been successful in creating both feasibility and active participant lists that were used to benefit trial execution within a large, integrated healthcare system. The success of this effort can largely be attributed to the expertise and capability of the VINCI group to both access and analyze the CDW data, as well as the incorporation of the overall NODES model which prioritizes collaboration and engagement of stakeholders at multiple levels within both CSP and VA, at large [37]. Additional work is needed to determine the feasibility of expanding this model to additional CSP trials and ultimately, non-CSP trials both within and external to VA. The research mission of VA is to improve the lives of Veterans and all Americans through healthcare discovery and innovation. As we make important advancements in the work we do in VA we turn to partners serving other Americans so that they can see the benefits of these advancements. The generalizability and sustainability of the model can be further examined once the model has been implemented in the aforementioned settings.

Disclaimer

The views expressed in this article are those of the authors and do not necessarily reflect the views or policies of the US Department of Veterans Affairs, or the US government. The mention of trade names, commercial products, or organizations does not imply endorsement by the US government.

Support

The research reported/outlined here was supported by the Department of Veterans Affairs, VA Office of Research and Development, Cooperative Studies Program (CSP) award N0009 and Health Services Research and Development Service (HSR&D), VA HSR RES 13-457.