Real-Time Enrollment Dashboard For Multisite Clinical Trials.

OBJECTIVE: Achieving patient recruitment goals are critical for the successful completion of a clinical trial. We designed and developed a web-based dashboard for assisting in the management of clinical trial screening and enrollment.
MATERIALS AND METHODS: We use the dashboard to assist in the management of two observational studies of community-acquired pneumonia. Clinical research associates and managers using the dashboard were surveyed to determine its effectiveness as compared with traditional direct communication.
RESULTS: The dashboard has been in use since it was first introduced in May of 2014. Of the 23 staff responding to the survey, 77% felt that it was easier or much easier to use the dashboard for communication than to use direct communication.
CONCLUSION: We have designed and implemented a visualization dashboard for managing multi-site clinical trial enrollment in two community acquired pneumonia studies. Information dashboards are a useful tool for clinical trial management. They can be used as a standalone trial information tool or included into a larger management system.

Background and significance

Clinical trials are necessary to test and validate new therapies and to advance the quality of care in healthcare. Consequently, the number of trials being conducted globally is increasing each year. A significant barrier to the successful completion of a clinical trial is failure to recruit the appropriate number of subjects. Common contributing factors to low trial participant accrual are short enrollment periods, insufficient numbers of staff available for recruiting patients, and disparate clinical research sites which are difficult to manage as a single unit; any of these problems can lead to a trial being terminated without gaining the knowledge for which it was designed [1].

Numerous approaches to improve patient recruitment have been proposed in the literature and in practice. Integrating trial recruitment more closely with patient care could reduce recruitment overhead [2], [3], [4]. Better access to data from previous trials could aid in planning successful future trials [5]. Many systems to streamline access to medical records for the purpose of recruitment have been proposed and implemented [6], [7], [8]. Efficient and accurate screening is also critical for the success of a clinical trial [9], [10], [11]. Systems for automated trial recruitment have been implemented along with systems specifically addressing time-sensitive recruitment [12], [13]. Clinical Trial Recruitment Support Systems have been designed to integrate clinical workflow with the entire clinical trial and regulatory process [14], [15], [16].

Recently, approaches to conducting multisite clinical trials have become an active area of research because workflows and methodologies for single site trials often do not scale, and systems for managing regional site networks allow for a broader patient pool to be used for recruitment [17], [18]. In addition, mobile and web-based technologies have become important in facilitating these efforts [19].

A tool for stakeholders to follow the progress of a trial could improve communication across multiple clinical trial sites encompassing multiple trial staff. A visual information dashboard coupled with the trial management system can provide this ability. An information dashboard is a visual representation of a body of information displayed on a computer screen. It is meant to show only the information most critical to a user's tasks and to provide timely updates to the information [20]. Some software products like Tableau can be used to help visualize clinically relevant data [21], [22]. Clinical scorecards, which arrange common qualitative clinical information in an easy to understand list format, are similar to dashboards and are used to provide clinical decision support [23].

Patient enrollment is critical to the success of a clinical trial, and an information dashboard can provide real-time data regarding enrollment to assist trial investigators, sponsors, and research staff in making timely decisions about recruiting policies and organizational decisions.

Objective

We developed a web-based clinical enrollment dashboard to provide clinical trial personnel with current information regarding enrollment progress for clinical trials. We identified three objectives for our system. The system must:

Compactly show current enrollment statistics for the study.

Allow clinical trial personnel to update and view enrollment numbers remotely

Display data in a way that allows easy comparison of enrollment across different sites

Materials and methods

An informational dashboard was designed to support two IRB approved observational studies of community-acquired pneumonia (CAP) managed by the University of Louisville Division of Infectious Diseases. Enrollment sites for both studies include all nine adult hospitals in Jefferson County, Kentucky. Clinical research coordinators working on the study screen, consent, and enroll patients seven days a week. The screening process began with coordinators accessing daily hospital admission records. Patients who meet study criteria were approached, consented, and enrolled into the study. Each hospital provided a partial privacy waiver for accessing medical records for the purpose of determining signs and symptoms consistent with CAP.

Needs assessment

Prior to developing the enrollment dashboard, our team conducted a needs assessment to determine the relevant information to display to stakeholders of the trial. The following items were selected for inclusion in the dashboard:

The number of patients enrolled per hospital per day. This serves as the primary means of quantifying the progress of enrollment.

The number of patients eligible to be enrolled per hospital per day. This number serves as the upper limit for possible enrollment.

The number of patients admitted to each hospital each day. This places the number of eligible patients within the context of a hospital's total admissions.

The number of patients screened per hospital per day. This highlights the efficiency of the screening process.

Name of person providing enrollment numbers at a given site. This provides a point of contact for enrollment numbers that may not match case report forms.

Requirements for the implementation of the system were also identified as follows:

Flexible software design to support changing project requirements.

Real-time capability.

Easy to develop and maintain.

In our view a web-based solution best satisfied these requirements; the following components were outlined for implementation of the system: a web server with a database to store enrollment numbers, an interface to allow enrollers to enter enrollment numbers remotely via smartphone or laptop, and a role-specific web page to display enrollment data as an information dashboard. To keep the system simple it only stores enrollment numbers, not any data associated with patients. This reduces the complexity, security, and privacy requirements of the software. Fig. 1 shows an example of the enrollment process using the software.

Fig. 1

Workflow diagram for patient screening and enrollment. A research associate performs patient screening by connecting to hospital EMR. Screening results are visualized on the dashboard. Research associates enrolling patients on site update enrollment numbers using a mobile device. Managers and Investigators can check dashboard for current enrollment progress.

Data storage

The backend of the enrollment dashboard uses the Linux-Apache-MySQL-PHP (LAMP) application stack. That is, a Linux-based operating system running an apache web server and MySQL database. A web browser is used to interact with the data using the PHP server-side scripting language. This is a popular backend setup, and there is a large selection of information on how to set up this stack quickly and easily [24]. We chose a cloud-based implementation of this stack as it provides the benefits of automatic scaling to handle traffic, and removes the space and maintenance overhead of using local equipment. Many cloud providers such as Amazon Web Services and Linode offer reasonably priced virtual servers with a LAMP stack configured and ready to deploy.

Mobile data entry

Each of the trial personnel possesses a smartphone capable of accessing the Internet through a cellular data plan or locally accessible WiFi. Consequently, we implemented a web-based method for staff using their personal computer or smartphone to enter data. We designed a user interface with the Bootstrap framework that allows a web page design to be optimized for display on either desktop browsers or smartphones. Fig. 2 shows an enrollment entry screen on a mobile device. OAuth, a protocol that allows users to access secure online content using third party login information from Google or Twitter accounts, provides authentication for the app. This allows coordinators to login using their Google account name. The data entry app verifies that the name and Google ID of the person logging in is a coordinator with access rights to the system. This type of authentication is especially useful on mobile devices where typing credentials can be challenging. Each coordinator was initially given a 1-h training session on how to enter enrollment data using the app. Additional questions were addressed at weekly meetings as the app began to be used in the field.

Fig. 2

Mobile data entry web page.

Enrollment dashboard screen

The information dashboard was designed to provide a visualization of enrollment data on a screen accessible to various clinical research personnel. The dashboard was implemented as a web app accessing a secure remote database and graphics were created using the d3 JavaScript library [25]. The primary graphics displayed by the dashboard are nested bar charts for each hospital showing total enrollment per hospital, shown in Fig. 3. Nested bar charts represent a group of bar charts that are overlaid to save space. This works well with daily enrollment visualization, as the bars will always be in descending order within a group. The largest bar will represent the number admitted to the hospital on a particular day, and the following groups will decrease in quantity for the number screened, eligible, and enrolled.

Fig. 3

Layered bar chart for enrollment data. The number of admitted, screened, eligible and enrolled is different for each site. Representing the graphs in as close a space as possible allows the progress for many sites to be displayed in one view as seen in Fig. 4.

Fig. 4

Enrollment dashboard showing daily enrollment numbers for the nine hospitals participating in the study.

Daily enrollment numbers are also available which show the complete statistics from the enrollment process. For each hospital in the study numbers are shown for how many patients were admitted, prescreened, screened, eligible, and enrolled for each study. The user responsible for uploading the data and update time is also shown.

Results

Dashboard use

The dashboard was made available to users on May 1, 2014 and has been in continuous operation since that time. A public facing version of the dashboard is provided to study sponsors so they can see the enrollment numbers at any time. Research investigators have access to a version that provides data divided by hospital and date, shown in Fig. 4. This data are used to quickly give an impression of enrollment for the two studies. This layout follows the principle of small multiples [26] whereby many similar charts are shown side by side to provide direct comparison. This layout could also be made to scale to a larger numbers of enrollment sites, where charts are grouped in a scrollable window.

Improvements to workflow

Research investigators are able to monitor the process of patient enrollment throughout the day. Common problems that occur in the process of having multiple clinical research associates are confusion about which hospitals have patients who need to be screened, which associates will be performing the screening and enrollment, which hospitals have lab samples ready to be retrieved, etc. Management procedures exist for each of these tasks, however there will be day-to-day difficulties that are not covered by policy. In these cases monitoring of the enrollment dashboard can provide increased awareness to clinical investigators and coordinators and improve workflow processes.

Feedback

Coordinators and managers were asked to complete an optional and confidential survey to assess their opinion of the dashboard's utility. The survey received approval from the Institutional Review Board and was hosted on SurveyMonkey.com. Twenty-three trial staff responded to the survey out of the 48 who were eligible. Seventy-seven percent felt the app made communicating screening and enrollment information easier or much easier than email or other types of messaging. Eighty-seven perccent felt that the app made the screening process more organized than management using direct communication. Results are shown in Table 1.

Table 1

Survey results from clinical research associates on dashboard utility. Responses were scored on a Likert Scale from Strongly Agree (5) to Strongly Disagree (1).

Question	Strongly agree (5)	Agree (4)	Neutral (3)	Disagree (2)	Strongly disagree (1)
Using the daily enrollment website is straightforward.	15	7	0	1	0
Compared to using email and messaging, using the daily enrollment website to communicate screening and enrollment numbers is easier.	14	4	3	2	0
Compared to using email and messaging, using the daily enrollment website makes patient screening more organized.	13	7	3	0	0
Compared to using email and messaging, using the daily enrollment website makes patient enrolling more organized.	12	6	4	0	1
The daily enrollment website has led to changes in enrollment and screening policy.	7	6	7	2	0

Survey respondents provided many useful recommendations in the open response questions, many of which are planned for the next iteration of the software. One of the most common requests was to allow the app to retroactively change the number of eligible patients after the patient was approached. Another common request was to have a figure that counts the number of patients who may be eligible but with pending consent, which could happen when the patient was not available to be consented at the time when the coordinator was present in the hospital or when patients were intubated and could not verbally consent. Additionally, it was suggested to stratify the number of unconsented into reasons for refusal, if given.

Discussion

As multi-site clinical trials become more common, so will systems for managing the business intelligence required to maintain trial efficiency. While current data collection and trial management solutions like OpenClinica and ClinCapture provide some dashboard functionality, these systems lack the ability to create customized dashboards for highlighting variables of interest or displaying visualizations that differ from the typical bar graph.

Open Source software has the advantage that it can be modified in any way by the users, but this is counterbalanced by the fact that the users must maintain the software and deploy it on their own webserver. Open source trial management software can be extended to include dashboard implementations like the one presented, but the programming experience needed will vary. The REDCap data collection software [27] is not open source, but does provide an accessible software structure that can be customized by end users. The database for REDCap can be accessed using a MySQL database interface, allowing data to be connected to customized information dashboards.

The clinical trial enrollment dashboard would benefit from many additional features. The ability to run reports that can be downloaded as well as the ability to stratify enrollment data according to factors such as country of enrollment or type of enrollment site is currently in development for the next iteration of the software. Also in development are visualizations to show trends for enrollment at each hospital. One approach is to use sparklines [28], which are small line graphs placed next to each site enrollment chart.

Limitations

While an extensive amount of programming expertise was not required for the creation of the system, adding more complicated features would increase the complexity and development time. Several features such as a comprehensive audit trail that tracks uploads to the system, user-customizable views, and report generation are currently in development.

Conclusion

An Information dashboard can be a useful tool to improve the conduct of clinical trials. We have designed and implemented a visualization dashboard for managing multi-site clinical trial enrollment in two community acquired pneumonia studies. Similar dashboards can be implemented as add-ons to existing systems or deployed as a separate web service using available data sources.