Comparison of Open-Source Electronic Health Record Systems Based on Functional and User Performance Criteria.

OBJECTIVES: Open-source Electronic Health Record (EHR) systems have gained importance. The main aim of our research is to guide organizational choice by comparing the features, functionality, and user-facing system performance of the five most popular open-source EHR systems.
METHODS: We performed qualitative content analysis with a directed approach on recently published literature (2012-2017) to develop an integrated set of criteria to compare the EHR systems. The functional criteria are an integration of the literature, meaningful use criteria, and the Institute of Medicine's functional requirements of EHR, whereas the user-facing system performance is based on the time required to perform basic tasks within the EHR system.
RESULTS: Based on the Alexa web ranking and Google Trends, the five most popular EHR systems at the time of our study were OSHERA VistA, GNU Health, the Open Medical Record System (OpenMRS), Open Electronic Medical Record (OpenEMR), and OpenEHR. We also found the trends in popularity of the EHR systems and the locations where they were more popular than others. OpenEMR met all the 32 functional criteria, OSHERA VistA met 28, OpenMRS met 12 fully and 11 partially, OpenEHR-based EHR met 10 fully and 3 partially, and GNU Health met the least with only 10 criteria fully and 2 partially.
CONCLUSIONS: Based on our functional criteria, OpenEMR is the most promising EHR system, closely followed by VistA. With regards to user-facing system performance, OpenMRS has superior performance in comparison to OpenEMR.

I. Introduction

Open-source software (OSS) is computer software with a licensing provision that enables the users to modify, utilize, and distribute unmodified or modified versions [1]. The free open-source software (FOSS) is also known as libre and open-source software (FLOSS), libre software, free software, and open-source software. In recent decades, the open-source business model has evolved, and FOSS is now more than just cost-free software. Because source code is available, FOSS has developed into many different distribution models [2]. OSS has several technical benefits, such as reliability, security, quality, good performance, flexibility of use, tester and user base, compatibility, and harmonization [3]. FOSS in healthcare is different because of the inherent need for domain-specific experts in the community. From 1999 to 2001, the healthcare industry saw the start of many OSS projects [4]. Several factors have contributed to the success of software products in healthcare. These include the existence of a user community and a developer community, regular release cycles, modular architecture, documentation of software, and support services [4].

One of the most significant benefits of OSS in the healthcare industry is that FOSS is easy to obtain as it can be downloaded easily from repositories. There are no restrictive licensing terms regarding distribution, derivative works, or modifications, and there are no software licensing fees. Open-source software is vendor neutral, which means that organizations or individuals do not suffer from vendor lock-in. Although there are numerous other benefits of cost savings and flexibility, opportunities to innovate while remaining independent from vendors is more applicable to healthcare [3].

Challenges of OSS in healthcare are organizational in nature, such as long-term support organization, funding and ownership for EHR certification, open-source business models, medical informatics and domain specific issues, functionality-based limitations, usability issues, challenges in resource-restricted environments, and challenges related human factors [5]. The main objective of this work is to guide organizational choice to select an open-source system for implementation by comparing the features and functionality of the five most popular open-source Electronic Health Record (EHR) systems using an integrated set of criteria. It might also provide useful insight for open-source developers to identify the functional requirements that are missing but required by clinical users. The user performance comparison results may be used by EHR developers to analyze and improve the usability of their products for basic tasks, such as patient search, ordering, viewing patient records, and so forth.

II. Methods

1. Information Source and Search

Relevant articles on open-source EHR systems were identified primarily using Scopus. We also used other databases, such as PubMed and IEEE Xplore, to ensure that we would not leave out any publications that are not indexed by Scopus. We searched publications from 2012 to 2017 to identify criteria to compare open-source EHR systems. Only English language articles were selected. The search terms were looked up in their titles and abstracts.

2. Study Selection, Data Collection Process, and Data Items

The next step was a detailed examination of papers and exclusion of irrelevant papers. We included articles that discussed ‘open-source electronic health record systems’ AND ‘comparison’ OR ‘adoption’ OR ‘utilization’ OR ‘analysis’ OR ‘application’. Figure 1 presents the PRISMA flowchart outlining the process for systematic review. Only full-text available articles were selected. Finally, we included a total of 25 articles.

Figure 1

PRISMA flowchart outlining the process of systematic review.

3. Literature Review

A study conducted by Zaidan et al. [6], evaluated various open-source EHR systems using multi-criteria decision-making of AHP-TOPSIS, which is complicated and requires an enterprise-wide understanding of the concept of an ideal solution. To assess OSS EHR functionality, we considered the following eight parameters: health information and data, results management, order entry and management, decision support, electronic communication and connectivity, patient support, administrative processes, and reporting and population health management. This is based on the Institute of Medicine (IOM), renamed as the National Academy of Medicine's eight-core features of an EHR system.

Goldwater et al. [7] studied the use of open-source EHR systems in chronic disease management. The study showed the positive impact of an open-source EHR system in improving diabetes and tuberculosis management and controlling hypertension. They identified six essential elements that should be available in an EHR system to support chronic disease management. These elements include modern clinical information services, systematic decision support systems, self-management support for patients, and links to community resources [7]. Meaningful use criteria include clinical decision support and availability of patient-specific resources; therefore, we included them in our integrated functional criteria [8].

Based on the results of the qualitative content analysis we selected meaningful use criteria and the eight-core functionalities from the IOM, finally totaling 32 criteria to compare the features and functionalities of various health FOSS. We adopted a set of performance criteria from the study conducted by de Abajo and Ballestero [9] to compare OpenMRS, OpenEMR, and VistA. Along with the literature, we thoroughly reviewed all the software documentation on the websites of the five EHR systems.

4. Comparison Criteria

The study was conducted as follows. The open-source EHR systems were compared using a set of performance-measuring parameters by deploying the software on a local machine. Then they were compared in terms of meaningful use criteria and the IOM core functionalities. Figure 2 shows an overview of the methodology.

Figure 2

Flowchart illustrating the methods of the current study. EHR: Electronic Health Record.

1) Identification and selection of open-source EHR systems

The open-source EHR systems were selected in two steps. The top five EHR systems were selected based on their global website ranking and identification of the reach and popularity of open-source EHR systems over time. Alexa is a global ranking system that utilizes web traffic data to compile rankings. From the 15 listed open-source EHR systems (See Table 1), VistA, GNU Health, OpenMRS, OpenEMR, and OpenEHR were selected based on their global rankings retrieved from Alexa rank. HOSxP changed its license in 2015, and we excluded it from our study due to its focus on a subscription-based hosted EHR. These five EHR systems vary greatly regarding their audience and the type of health facility that they target. However, we compared the clinical features that should be present in all EHR systems, irrespective of whether they are implemented in a primary care clinic, community hospital, or a tertiary hospital, even though the sophistication of those functionalities might differ based on needs. OpenEHR is a standard, a set of archetypes that are implemented in EHR system products. OSEHRA VistA and GNU Health are hospital information systems. OpenEMR and OpenMRS are EHR systems or platforms, respectively, but those are details that are common across proprietary EHR systems, which market their products to different audiences and focus on certain types of clinical settings. A Google Trends comparison of the five selected EHR systems from August 2017 to August 2018 is shown in Figure 3.

Table 1

List of open-source EHR systems along with the official website and its global rank

EHR: Electronic Health Record.

Figure 3

Comparison of interest over time (August 2017 to August 2018) for the five EHR systems. EMR: Electronic Medical Record, EHR: Electronic Health Record, MRS: Medical Record System.

Figure 4 shows the regional popularity of each of the five EHR systems. It is interesting to note that OpenEHR is clearly more popular in Northern European countries, Scandinavia, Brazil, and Australia than the rest of the world. GNU Health has some interest in China, the US, Argentina, Germany, and Spain. OpenEMR and OpenMRS have more global reach, and this can also be identified by consideration of the global implementation map of these EHR systems. OpenMRS is clearly very popular and widely implemented in Africa, India, and South East Asia. OpenEMR has implementations in India, the United States, Brazil, the UK, and South Korea. VistA is primarily used by the Veterans Affairs (VHA) in the United States, with some custom implementations and distributions of it in other places.

Figure 4

Interest of the five selected EHR systems by region. EMR: Electronic Medical Record, EHR: Electronic Health Record, MRS: Medical Record System.

2) Evaluation

The evaluation in our study was carried out in two main phase, one by deploying the software and comparing the EHR systems using the standard set of criteria, and subsequently by comparing the performance parameters.

(i) Deploying each open-source EHR software: Documentation available on respective EHR system official websites were used for deployment.

(ii) Establishing criteria to evaluate and compare: Based on the literature review summarized above, using qualitative content analysis, we developed an evaluation criterion extracted from the meaningful use criteria and eight-core functionalities listed by the IOM [10].

3) Performance criteria

The selected parameters of user performance criteria include initial CPU use, startup time of the server, shutdown time of server, login time and CPU use, patient search time and CPU use, time for creating new patient and CPU use, time for change of patient and CPU use, average navigation time for the main menu and CPU use, logout time and CPU use. These performance criteria were implemented on a machine with a 64-bit Windows 10 operating system and 8 GB RAM.

III. Results

The interoperability comparison focused on criteria based on meaningful use presented in Table 2.

Table 2

Findings of interoperability-focused criteria

EMR: Electronic Medical Record, EHR: Electronic Health Record, MRS: Medical Record System.

1. Transition of Care and Clinical Information Reconciliation

ZH Healthcare incorporated a mechanism to upload, process, display, and incorporate CCDA into OpenEMR. In OSEHRA VistA, this criterion is fulfilled by popHealth, which is an open-source quality measure reference implementation. In OpenEMR clinical information, the reconciliation criterion is fulfilled by the ZH Healthcare coordination module. OpenMRS stores all of a patient's details, visits, encounters, and observations.

2. Patient-Specific Education Resources and e-Prescribing

This is available only in OpenEMR and OSEHRA VistA. In OpenEMR, this feature is available under the patient support tab in the user interface, and in OSEHRA VistA, this criterion is fulfilled by infobuttons. In OpenEMR (see Figure 5), e-prescribing can be done in three ways: Newcrop (a subscription service integrated with OpenEMR), AllScripts (it does not have fees, but it focuses on the US phamacy networks), and Weno (another US-centric, free eRx routing service). On the other hand, GNU Health (see Figure 6) has a very local formulary-focused workflow, where all drugs have to be entered manually, and prescriptions are made from those. In OpenEHR, this criterion is fulfilled under a location-independent medication management feature. In the case of VistA, it has a separate e-prescribing module.

Figure 5

Patient education with messages and reminders in Open Electronic Medical Record (OpenEMR).

Figure 6

Generation of e-prescriptions in GNU Health.

3. Incorporation and Transmission of Electronic Laboratory Tests

In OpenEMR, this has been supported by the ZH Healthcare (an EHR vendor) laboratory module. In OpenMRS and GNU Health, laboratory results are stored, but not in accordance with the HL7 implementation guide. In OSEHRA VistA, this criterion is fulfilled by the VistA laboratory module. Transmission of electronic laboratory tests is possible in all the selected systems. In openEHR, laboratory results can be transmitted to other EHR systems. In the case of VistA, this criterion is fulfilled by the VistA laboratory module.

4. Data Portability and Clinical Quality Measures

In openEHR, this is supported by the ZH Healthcare care coordination module. OpenMRS exports according to the Clinical Document Architecture (CDA) standard, not the CCDA (a US-specific standard Consolidated CDA) standard. In VistA, data portability is fulfilled by a separate but related product called popHealth. OpenEMR supports capture, calculation, import, export, and electronic submission of clinical quality measures. OpenMRS, GNU Health, and OpenEHR calculate quality indicators only. In VistA, the CQM criterion is fulfilled by popHealth.

A comparison of the privacy- and security-focused meaningful use criteria are presented in Table 3.

Table 3

Privacy- and security-focused meaningful use criteria

EMR: Electronic Medical Record, EHR: Electronic Health Record, MRS: Medical Record System.

5. Authentication, Access Control, and Authorization

OpenEMR supports security measures, such as the authentication mechanism, access control, and authentication. In OpenEHR, this provided by Inidis, which is one of the OpenEHR partners. GNU Health has a central authentication system, and access control is fulfilled by security features in GNU Health. VistA has a strong access management architecture.

6. Automatic Log-Off and Integrity

In OpenEMR, the automatic log-off feature was tested and confirmed by Jones et al. [11]. In VistA, the auto log-out feature can be enabled by applying a few settings on the VA user profile. Integrity is implemented in OpenEMR as part of the Certification Commission for Healthcare Information Technology (CCHIT) requirements. In OpenMRS, the integrity of data is ensured by the data integrity module. In OpenEHR, integrity is provided by the versioning model specified in the change_control package.

7. Order Entry/Order Management and Decision Support

OpenEHR, OpenEMR, and VistA allow computerized provider order entry. In OpenEHR, clinical decision support is delivered through guideline definition language. GNU Health also uses the World Health Organization essential list of medicines. In VistA, decision support is supported by infobuttons.

8. Electronic Communication, Connectivity, Patient Support and Reporting for Population Health

Electronic communication between providers is available only in OpenEMR, OpenMRS, and OSEHRA VistA. In OpenEMR, patient-specific resources can be found under the patient support tab in the user interface. In VistA, patient-specific resources are supported by infobuttons. OpenEMR reports immunization information, syndromic surveillance, and cancer cases to public health agencies. Reporting of population health is done under the notifiable condition detector module. Table 4 compare the EHRs based on IOM's feature set.

Table 4

Institute of Medicine's core features criteria

EMR: Electronic Medical Record, EHR: Electronic Health Record, MRS: Medical Record System.

IV. Discussion

In this study, we compared the selected systems using the integrated criteria that were developed. Although a few studies have compared the EHR systems [121314] that use a few criteria based on meaningful use, none of them have considered all the criteria listed by the Office of the National Coordinators for Health IT (ONC).

1. Comparison with Meaningful Use Criteria

In the context of health information technology, meaningful use is defined as the minimum US standard required for using EHR systems and for exchanging information between providers, between providers and insurance companies, and between providers and patients. While we acknowledge that other countries, particularly low- and middle-income countries [1516] might not legislatively require all of these features, their importance in all clinical settings cannot be ignored. Except for OpenEMR, none of the other selected systems have been verified for ONC's meaningful use criteria, though OSEHRA VistA has most of the criteria fulfilled, and some vendor distributions of VistA have been verified.

Limitation: More than 15 open-source EHR systems are available, but because of time constraints, we compared only the five most popular EHR systems.

2. Core Functionalities

OpenEMR and OSEHRA VistA have all the eight-core functionalities, while other systems are lacking a few. Health information and data, result management, decision support, and administrative processes are the functionalities available in all the selected EHR systems.

Limitation: Because of the unavailability of web-based demo databases for GNU Health, VistA, and OpenEHR, we had to deploy locally based on publicly available configuration documentation. We acknowledge that some vendors might deploy these systems differently. Also, certain EHR systems are specially or purposefully built for epidemics or disaster situations [17]. We ignore these as edge cases, and only consider general-purpose EHR systems.

3. Performance

The top two performing systems were OpenMRS and OpenEMR. The GNU Health, VistA, and OpenEHR-based systems were more complex and required many customizations before we could execute the user performance tasks. OpenMRS was found to be faster than OpenEMR. The CPU utilization is also higher with OpenEMR, as shown in Table 5.

Table 5

User performance criteria

EMR: Electronic Medical Record, MRS: Medical Record System.

Limitation: The performance analysis was done on a single system, and this may differ from the performance of a system in a real-world setting. However, our goal was to test user performance on the most regular EHR tasks.

4. Documentation

OpenEMR, OSEHRA VistA, OpenMRS, and GNU Health have good documentation. OSEHRA VistA has a very detailed package list including their descriptions and implementation guide.

Limitation: The documentation for OpenEHR is very poor, which forced us to rely on other web resources and articles to support the availability of features.

5. Conclusion

Investigating available open-source EHR systems is necessary, but few studies have compared the existing open-source EHR systems. These studies on available open-source EHR systems are not exhaustive, especially on security and interoperability. In this study, the top five globally ranked EHR systems were compared using integrated criteria that we developed. Based on our findings, in terms of functionality, OpenEMR is the most promising EHR system, closely followed by VistA. Other systems were found to be lacking in a US-specific functional requirement, but also in some security and interoperability requirements. Regarding performance on the most common user tasks, OpenMRS achieves superior performance in comparison to OpenEMR.