Roadmap to a more useful and usable electronic health record.

Background: A decade after the Health Information Technology for Economic and Clinical Health (HITECH) Act, electronic health records (EHRs) largely remain poorly designed and contribute to clinician burnout. Objective: The purpose of this study was to understand clinicians' wants, needs, and perceived barriers imposed by the EHR; implement best practices in user-centered design; and create a clinician-centered EHR framework validated via a functional EHR prototype.
Methods: Usability evaluations were performed using a simulated patient with a complex clinical scenario. Convergent parallel mixed methods linked to action research and agile development were used to create an EHR prototype based on clinician-centered design. Prototype functionality was validated via a final usability evaluation.
Results: Between 2015 and 2017, 53 clinicians from 8 cardiology practices (4 academic and 4 private) participated in initial evaluations of their installed EHR. In 2019, 25 clinicians participated in final evaluations of their EHR vs our EHR prototype. Initial evaluations documented that clinicians judged the EHRs as poorly designed, scoring a mean of 47.1 on the System Usability Scale. Clinicians expressed that EHRs impeded workflow and communication and prolonged their workday. In the final evaluations, no improvement in installed EHRs was found (mean score 48.1); however, the EHR prototype was assessed as significantly more usable (mean score 77.8; P <.001).
Conclusion: A decade after the HITECH Act, EHRs still receive low usability scores. By applying user-centered design, an EHR prototype with improved features, functionality, and workflow integration was developed. Clinician testing of the EHR prototype demonstrated it was significantly more useful and usable to clinicians, thus identifying a framework and pathway for substantive improvement of EHR systems.

Use of a simulated patient and a complex clinical scenario captures clinician wants, needs, and barriers imposed by the electronic health record (EHR). This secure, Health Insurance Portability and Accountability Act–compliant method allows uniformity of data collection across multiple sites and multiple installed EHRs.

The use of parallel convergent mixed methods (quantitative, qualitative, and heuristics) linked to agile development and user-centered design allows the generation of novel usability solutions that can be adopted by EHRs.

The clinician-centered EHR prototype, as measured by qualitative and quantitative analyses (the System Usability Score) is more useful, usable, and satisfying to clinicians than their installed EHR. In particular, building data collection into the workflow; linking diagnostics, therapeutics, and quality to specific problems; and pushing those data elements to clinicians improved usability.

Background

Widespread adoption of the electronic health record (EHR) was touted by many as a panacea to fix health care.1, 2, 3, 4, 5, 6 Following an auspicious start catalyzed by the creation of the Office of the National Coordinator for Health Information Technology (ONC) and the passage of the Health Information Technology for Economic and Clinical Health (HITECH) Act, health care is now facing the actualization of Merton’s “unanticipated consequences.” Instead of driving clinician enablement, the EHR is a major contributor to clinician dissatisfaction and burnout. Studies document that clinicians spend as much or more time with their computer than their patients.10, 11, 12 This situation has been attributed to poor EHR usability and administrative burden., In response, major professional societies, and even ONC have released policy statements and recommendations for improving the EHR.

In 2014, with support from the Agency for Healthcare Quality and Research (HS022110-01A1), we initiated a multisite, multivendor project with 3 objectives: (1) understand the wants and needs of, and perceived barriers imposed by the EHR on, clinicians; (2) apply best practices in user-centered design and action research methodologies to create a clinician-centered EHR framework; and (3) validate the framework via a functional EHR prototype.

Usability in health information technology has been defined as either efficient, effective, and satisfying or useful, usable, and satisfying. Our foundational premise is that improving on these definitions of usability can be achieved by applying the principles and techniques of user-centered design, linked with action research and agile development, to construct an EHR framework that better meets the needs of clinicians. With this approach, clinicians and designers iteratively work in tandem to define the solution set that fits the user’s mental model and workflow of tasks and outcomes.

Methods

The project was divided into 3 phases: (1) an initial evaluation to understand the wants and needs of clinicians, as well as the perceived barriers imposed by their EHR; (2) user-centered design—applying heuristic analysis, action research techniques, and agile development to incorporate desired functionality into an EHR prototype; and (3) testing and validating a functional EHR prototype compared to the clinician’s installed EHR. Participation in the initial evaluation was not a requirement for participation in the final evaluation.

Clinicians, defined as physicians or advanced practice providers (APPs) who conduct assessments and create the care plans for patients, were recruited from 8 institutions. The 4 academic sites were Duke University (Durham, North Carolina); Indiana University (Indianapolis, Indiana); University of Nebraska Medical Center (UNMC) (Omaha, Nebraska); and the Veterans Administration Center in Omaha, Nebraska (affiliated with Creighton University). The 4 private practice sites were Faith Regional Medical Center (Norfolk, Nebraska); Swedish Medical Center (Seattle, Washington); Parkview Health (Fort Wayne, Indiana); and Ascension Health (Indianapolis, Indiana). The study was approved by the University of Nebraska Institutional Review Board (IRB). Individual site IRB approval was obtained when requested. Privacy and security concerns at each site were addressed before testing. Research subject selection was at the discretion of the individual study site. The research team had no role in subject recruitment. This research report conformed to the Helsinki Declaration guidelines as revised in 2013.

Initial evaluation: Understanding the wants and needs of clinicians

To understand the perceived barriers to efficient and effective care, and the wants and needs of clinicians, a multidisciplinary design team of clinicians and usability experts applied parallel convergent mixed methods24, 25, 26 using a complex clinical scenario and a simulated patient (actor)., Parallel convergent mixed methods are a pragmatic approach that combines qualitative and quantitative methods to facilitate a more complete understanding of problems. Qualitative analysis looks for concordance of themes determined by the frequency of statements, the convergence of different subjects, and the intensity of their statements. Additional research subjects were added until saturation of statements was achieved. Qualitative analysis has the ability to identify themes and new insights.

Use of a simulated patient addressed privacy and security concerns and allowed standardized evaluations across multiple sites and multiple EHRs. To validate the realism of the simulated patient and clinical encounter and to enable clinicians to fully express their thoughts, an exit survey using a 5-point Likert scale was conducted.

Cardiovascular disease was chosen for the broad spectrum of clinical contexts (from benign to life-threatening) and care settings (ambulatory, emergency department, and inpatient) represented. In additon, cardiovascular medicine has established robust quality metrics.

After obtaining informed oral and written consent, participating clinicians (research subjects) completed a brief demographic survey and the 100-point System Usability Scale (SUS) assessment of their installed EHR., The SUS survey was chosen because it is a validated standard for assessing EHR usability. A single-item Satisfaction Score using a 5-point Likert scale also was administered. After completion of the clinical scenario, clinicians were asked to express their wants and needs of EHR systems and to explain barriers they identified with use of their EHR.

The prescribed scenario was a patient who had just moved to town and was establishing care with “your” cardiology practice. The clinician was given paper-based records from an outside facility to review. The clinician then interviewed the simulated patient, created an assessment, and documented the assessment and plan.

The sessions were independently observed by 3 to 4 members of the research team, with the sessions recorded and transcribed to facilitate qualitative analysis. During each session, 1 observer (JW) scored the inclusiveness of the 21 key quality metrics (Table 1) of the American College of Cardiology’s PINNACLE (Practice INNovation And CLinical Excellence) Registry.

Table 1

PINNACLE quality metrics

Coronary artery disease	Heart failure	Atrial fibrillation	hypertension
History of myocardial infarction	New York Heart Association functional class	Stroke risk calculated	Hypertension medication prescribed
Presence of a coronary stent	Ejection fraction	Antithrombotic therapy	Blood pressure at target
Antiplatelet therapy	Heart failure symptoms	Symptom assessment
Statin therapy	ACE/ARB/ARNI therapy
Beta-blocker therapy	Beta-blocker therapy
Canadian Cardiovascular Society (CCS) Angina Score	Implantable defibrillator counseling
Smoking status	Heart failure education
Exercise prescription

Quality metrics from the American College of Cardiology’s PINNACLE (Practice INNovation And CLinical Excellence) Registry were used to measure effectiveness. These 21 elements represent the metrics for the 4 primary problems addressed during the clinical encounter.

ACE = angiotensin-converting enzyme inhibitor; ARB = angiotensin II receptor blocker; ARNI = angiotensin receptor neprilysin inhibitor.

User-centered design: Applying heuristic analysis, action research techniques, and agile development to incorporate desired functionality into an EHR prototype

The design team convened weekly to review and analyze data from the individual initial evaluation sessions. To facilitate innovation and provide independent validation, we impaneled the American College of Cardiology Informatics and Health Information Technology Taskforce (ACC Taskforce) to participate in the action research and agile development. An independent facilitator utilized ThinkTank (GroupSystems Corp., Denver, CO) to perform Delphi modeling and consensus building through complete and anonymous input. Under the guidance of the facilitator and input from the ACC Taskforce, the design team deconstructed the clinical encounter into core concepts and clinical tasks. The clinical encounter was then reconstructed with key design drivers used to create a fully functional EHR prototype.

The EHR prototype was developed using Bootstrap and Angular as the graphical user interface and Microsoft SQL as the database, with Java Spring Boot serving as an API/middle layer.

Final evaluation: Testing and evaluating the EHR prototype

For the final evaluation sessions, a new simulated patient case study was created. In this simulated case, the patient came to “your” emergency department and was admitted with a complicated acute myocardial infarction. The patient is now transitioning to your ambulatory practice.

After obtaining oral and written informed consent, we repeated the SUS evaluation of the clinician’s installed EHR system. This enabled assessment for evidence of improvement of the installed EHR compared with the initial evaluation and anchored the comparison with the EHR prototype. The installed EHR was the same system at all sites.

For evaluation of the EHR prototype, participants were instructed on the design principles and functionality (training typically required 15–20 minutes). As in the previous simulation, the clinician reviewed the records and then interviewed the patient, created an assessment, and documented the encounter using the EHR prototype. Prototype validation involved a debriefing of each research subject after the simulated encounter with qualitative interviews and a quantitative SUS assessment and single-value Satisfaction Score assessment.

Qualitative and quantitative analyses

Qualitative analysis of the session recordings was performed using NVivo software (QSR International, Burlington, MA). Two reviewers independently reviewed session transcripts, identifying and scoring themes for frequency, intensity, and convergence. Disagreements in themes were adjudicated by a third reviewer until consensus was achieved. The themes were validated by consensus of the ACC Taskforce. Quantitative analysis was conducted using the Student t test to compare initial and final EHR assessment scores, and a paired t test to compare final EHR assessment with the EHR prototype. Numerical data are summarized as mean ± SD.

Results

Between 2015 and 2017, 53 clinicians (19 female, 34 male) from 8 sites were surveyed. Participants included 28 practicing cardiologists, 12 fellows in training, and 13 APPs. Fourteen reported EHR use between 10–20 hours per week, and 39 reported >20 hours per week. All participants had >6 months of experience with the installed EHR. Forty reported extensive experience with Epic, 13 with Computerized Patient Record System (CPRS), 9 with Cerner, 4 with Allscripts, 4 with NextGen, and 4 with Athenahealth EHR systems. Three sites (UNMC, Duke, and Parkview Health) also had experience with homegrown electronic medical record systems before the EHR Incentive Program of the HITECH Act. To reduce potential bias, we excluded SUS scores from clinicians at UNMC and those who served on the ACC Taskforce (n = 14). Thus, 39 clinicians participated in the quantitative data collection.

Qualitative results

Participating clinicians across all sites were asked to identify barriers to efficient and effective care and their wants and needs to overcome those barriers. There was convergence across sites and installed EHRs that the installed EHRs impede clinician workflow, inhibit communication, and adversely affect decision-making. Furthermore, despite training, EHR designs were not intuitive, having too many pages and requiring too many clicks. Clinicians felt that reviewing patient records and documenting patient encounters were highly burdensome. This burden related back to too many administrative tasks that clinicians felt did not directly relate to patient care, what more than one clinician referred to as “documenting impertinent negatives.” A common comment across sites was that the EHR added 90 minutes to the workday. Specific comments across sites included the following: clinical notes were bloated and hard to read; problem lists were not well curated and were burdensome; and copying and pasting was frequently mentioned as a method to efficiently bring forward information from previous encounters.

The theme of wants also was consistent across sites. Clinicians stated they wanted high-quality, context-specific, verified data precompiled and pushed to them, along with easy access to good patient narratives. They wanted intuitive support for documentation and ordering, and a reduction in the length of notes. Of note, the wants, needs, and barriers were consistent not only across sites but also across clinician licensure and training.

Quantitative results

The design and usability of the installed EHR to manage the complex clinical scenario and simulated patient were assessed using the SUS. During the initial evaluation, clinicians rated their installed EHR design as poor (47.1 ± 16.8; range 20–82.5) and their overall satisfaction with the EHR as neutral (3.1 ± 1.0).

Effectiveness was measured using quality metrics from the PINNACLE Registry (Table 1). The mean score was 15.4 (range 8–21). This project specifically did not analyze the effectiveness of individual clinicians; instead, the intent was to identify consistent gaps across clinicians in terms of the quality of documentation. The most commonly missed measures were the Canadian Cardiovascular Society angina classification, New York Heart Association heart failure classification, and stroke risk conferred by the CHA2DS2-VASc score. We also identified the prescription of exercise and treating hypertension to goal as common issues not addressed in this simulation.

All 39 clinicians completed a 2-question postprotocol survey to validate our methodology. The simulated patient and the complex clinical scenario did accurately reflect their clinical practice (4.6 ± 0.4) and allowed them to fully express their wants, needs, and barriers to effectively using an EHR (4.8 ± 0.2).

To address the barriers imposed to efficient and effective care and to meet the wants and needs of clinicians, the design team, in conjunction with the ACC Taskforce, approached the solution with a clean slate. This required breaking the clinical encounter into its core components and mapping the tasks associated with the clinical encounter.

The task analysis is given in Table 2. The tasks are categorized as direct patient care, administrative data collection, medical decision-making, and communication. Across different organizations, we noted different roles and responsibilities assigned to physicians, APPs, the clinical team, the primary care provider, and even patients. Although centers differed in task assignments, no differences in the tasks needed to complete the clinical encounter were identified. Consistent across all organizations were the clinician’s core responsibilities to review the data and documents, interview the patient, synthesize history and diagnostic data to form clinical diagnoses, and formulate and document a plan including orders.

Table 2

Deconstruction of the clinical encounter into a tasks matrix

Direct patient care	Administrative data collection	Medical decision-making	Clinical documentation and communication
Chart review	Demographics	Problem list reconciliation	Self and partners
Obtaining history	Quality metrics	Medication reconciliation	Clinical team
Reviewing diagnostics	Registries	Decision support	Primary care provider
Information synthesis	Research	Orders	Patient
Patient education	Billing	External references	Payer
Patient engagement

Deconstruction of the clinical encounter resulted in the identification of 4 participants in the clinical encounter workflow: the clinician, the clinical team, the primary care provider, and the patient. Tasks identified as part of the clinical encounter occurred under 4 major headings: direct patient care, administrative data collection, medical decision-making, and communication. Optimizing workflow requires an understanding of what different participants can and should do.

Many of the components of the clinical encounter are remnants of paper-based charting and the classic history and physical examination. Using the methods described previously, we reframed the clinical encounter through a modern lens. We identified 4 core concepts from the clinical encounter that helped inform our design:

Concept 1: Episodes of care are an artifact of paper-based charting and billing. Patient care is a continuum, with episodes of care representing a snapshot of a point in time.

Concept 2: The physical examination is no longer central to the clinical encounter. A better formulation is history and diagnostics, with the physical examination a component of diagnostics. The history is obtained from 2 sources: the EHR and the patient.

Concept 3: For the clinician, the encounter is broken down into 3 components: review, interview, and document. The review and interview processes are iterative, not sequential. Clinicians synthesize the data to form a mental model of the patient and the patient’s problems.

Concept 4: Clinicians want easily searchable data, not voluminous documents. The problem list, if appropriately curated, serves as the anchor for efficient and effective use of the EHR.

Once the core concepts and tasks associated with the clinical encounter were established, we worked to optimize the usability of our EHR prototype. The classic definitions of usability describe what usability is, but not how it is obtained. Therefore, we chose to use a functional definition of usability (ie, optimizing workflow and dataflow, and reducing the cognitive load imposed by the EHR). Although we were able to observe the workflow at each of the 8 sites, optimizing workflow through task assignment optimization was beyond the scope of this project.

To increase patient-clinician contact time, gains in efficiency must come by reducing the time needed to review and document the clinical encounter. Core to our design is transitioning away from documents in favor of data. Data can be aggregated and viewed as needed. Furthermore, the prototype utilizes the problem list and problem-based connectors as proposed by Lawrence Weed to link data to actions., The prototype associates candidate diagnostic, therapeutic (including medications), and quality metrics for each clinical problem. These connectors are intended to facilitate both documentation and ordering. Previous work identified that different clinicians want and use different data.

We constructed a clinical encounter framework for dataflow (Figure 1). Data obtained through both history and diagnostics are collected, synthesized, stored, and retrieved. These 4 steps create a closed loop that fully informs the clinical encounter and is consonant with our task analysis (Table 2).

Figure 1

Data framework based on clinician-centered design. Clinical encounter dataflow deconstructed into components of data collection, data synthesis, data storage, and data retrieval. How data can be mapped to workflow and tasks is shown.

The next step in data optimization is to connect the pertinent data from previous clinical encounters through the lens of data persistence. Two exemplars of data persistence are fever and heart transplant. A temperature is an important datapoint, and an elevated temperature (fever) may contribute to formulating a diagnosis. However, fever, in decision-making, is very transient. In contrast, a heart transplantation persists and continuously informs decision-making. Therefore, the artifacts of past medical history, past surgical history, family history, and social history exist because they too inform clinicians of potentially relevant historical events. Because our framework promotes data over documents, the act of verifying data replaces the act of copying and pasting notes. Efficiency and effectiveness are facilitated by refreshing data through the lens of data persistence (Figure 2).

Figure 2

Following the initial encounter incorporation of data persistence and appropriate verification creates a simplified workflow and dataflow for the clinician.

At each of the 8 sites, we noted clinicians struggled to navigate through the EHR to find needed data and information. Thus, the third pillar of improving usability was understanding and optimizing cognitive load. Sweller states, “cognitive load theory consists of aspects of human cognitive architecture relevant to instruction along with instructional consequences that flow from the architecture.” Cognitive load can be intrinsic to the task or extrinsic such as that imposed by an EHR. Linking the functional definition of usability with the core concepts and task analysis was incorporated into a series of design assumptions that drove our prototype design:

Design assumption 1: Cardiovascular medicine is practiced the same across the country and independent of installed EHR. Although different individuals adopted different workflows and processes during the clinical encounter, there was little variance in tasks and desired functionality across sites. This was confirmed by Delphi modeling with the ACC Taskforce.

Design assumption 2: Clinical care is continuous. An encounter simply represents a snapshot of the patient by a clinician at a specific point in time. All of the patient’s medical history should be represented within the EHR. To organize clinical content, we created the metaphor of the patient’s medical record as a library; different clinicians want different “books” from the patient library. The clinician can take a book off the shelf, view it, use it, or reshelve it. This metaphor reduces note bloat while supporting information synthesis and documentation for billing.

Design assumption 3: EHR data can exist in 3 states: collected, clarified, and verified. In addition to the data framework illustrated in Figures 1 and 2, another step to optimizing clinician workflow is the inclusion of the patient in data collection. However, because of varying levels of medical knowledge by the patient, there is a need for data clarification (appropriate translation of patient terminology into medical terminology). In this framework, any member of the health care team can clarify data. The final step is data verification, a step that is the responsibility of the clinician provider.

Design assumption 4: Clinicians want pertinent data pushed to them. Using Weed’s concept of the problem list and problem connectors, it is possible to push pertinent data (including quality metrics and images) to the clinician. With the help of the ACC Taskforce and use of Delphi modeling, we were able to determine what information clinicians wanted pushed to them for specific cardiovascular disorders. This included embedded reminders and context-specific decision support.

Design assumption 5: It is important to minimize the extrinsic cognitive load imposed by the EHR while supporting clinician expertise. The heuristics of good design to reduce cognitive load are well established.36, 37, 38 The consistent use of well-established conventions and actions across all web pages is important to reduce the cognitive load imposed by an EHR. Although humans have a limited ability to hold multiple independent data elements in short-term memory, studies of experts recognize their ability to process chunks of data and place them in established schema to provide greater granularity of data and identify narrative gaps. Furthermore, we found that standardized actions portrayed across a large physical display with no hidden data optimized data representation and recall. Of note, although cited as a major factor in clinician burnout,, we specifically incorporated all known administrative tasks within the fabric of our prototype design.

Heuristic analysis coupled with action research and agile development pointed to the benefits of large, high-resolution screens. This allowed for greater display of data with a substantial reduction in clicks. Thus, our EHR prototype used 2 high-resolution (4K) monitor screens. Reviewing and validating data is predominately a function of the left monitor screen. Information synthesis is facilitated by keeping key data and narrative persistent on the screen. The library metaphor is used to review clinical data and the building of different views of the data on the right monitor screen.

Figures 3, 4, and 5 show snapshots of the full implementation of the tasks, concepts, and observations into a unified, comprehensive framework represented by the EHR prototype. The first step is to push domain-specific content (in this case cardiology) to the front. Although all of the patient’s medical record is available and readily accessible, we found that clinicians, whether in private practice or academics, and whether an APP or a physician, wanted the same cardiology-specific data pushed to them. The quick tabs give clinicians quick access to all data and a visual clue to new data. Perhaps more importantly, information remains readily accessible when clinicians want to refresh their mental model of the patient’s conditions.

Figure 3

Prototype electronic health record (EHR) utilized 2 high-resolution screens. The left-hand screen (shown here) is dedicated to information review to help the clinician build the patient mental model. Clinicians want domain-specific views. In Circle 1, the cardiovascular view is selected. The top bar is for ready access to current and previous outpatient and inpatient visits along with a quick review menu (Circle 2) that stays visible and active throughout the encounter. Problem lists drive associated problem connectors for imaging, labs, and quality measures (Circle 3). In this case, 3 of 5 quality indicators have already been recorded, and 2 need to be completed. In our framework, data exist in 3 states: collected, clarified, and verified. Verification is the responsibility of the treating clinician. The Command button for clinicians to verify data, using a “one click,” is shown in Circle 4.

Figure 4

Example of results based on user-centered design. Traditionally, medications are listed in alphabetical order. This is not consistent with the mental model and workflow of clinicians. The prototype linked (and displayed) medications to problems. Clinicians felt that this improved their ability to readily access compliance with quality measures and best practices. Not shown is the ability to present medications linked to problems over time. Clinicians felt this would reduce their time searching the electronic health record for information and avoid repeating previous failed therapies.

Figure 5

Left-hand screen. Clinicians wanted the ability to view (Circle 5; located in the bookshelf), verify, and annotate diagnostic data (Circle6; and, if desired, move that information into the note [Circle7]). The bookshelf (bottom line) stores all data verified in the encounter, while the note (right-hand column) brings in only data that the clinicians want for information synthesis and communication (shown in Circle 8).

The use of problem-based connectors seems to significantly reduce cognitive load of linking labs, diagnostic and therapeutic data, and quality indicators. This is particularly true with medications (Figure 4). Medications being linked to patient problems supports clinician data synthesis and identification of care gaps. The ability to retrieve previous problem-specific medications over time was viewed as very useful.

The importance of intelligently pushing data forward is shown in Figure 5. Whereas APPs were often satisfied with procedure reports, cardiologists appreciated access to raw data such as cineangiography or the cardiac magnetic resonance image (Figure 5). Clinicians liked the ability to bring in their interpretation of the data with or without the full report. The ability to document all of the data evaluated in the patient’s care but to include only the data necessary for information synthesis and communication was viewed as desirable.

Between May 2019 and November 2019, 25 clinicians from 7 sites participated in validation of the prototype (UNMC and ACC Taskforce members were excluded to reduce bias). We tested 15 practicing cardiologists, 3 fellows, and 7 APPs. The installed EHR system was the same as the EHR system at the initial evaluation across all sites.

Clinicians noted significant gains in efficiency and effectiveness through the use of a logical layout to the 2 screens, incorporation of data persistence, the ability to review raw data and images, and facilitation of data review and documentation by having the patient and nursing staff collect and clarify data, with validation accomplished with a simple click. Although concerns about the utility and curation of the problem list were voiced, clinicians understood and appreciated having the EHR compile and aggregate data while anticipating diagnostic testing and therapeutic intervention recommendations.

The SUS scores of the installed EHRs remained poor at follow-up (48.1 ± 16.7; range 27.5–92.5) and were essentially unchanged from the baseline obtained several years earlier (2015–2017). In contrast, our prototype scored 77.8 ± 12.4 (range 52.5–92.5; P <.001), a value at the upper range of good, bordering on excellent (Figure 6). Single-value Satisfaction Scores also reflect this difference. The follow-up Satisfaction Score for the installed EHR was 3.2 ± 0.9 vs the prototype score of 4.4 ± 0.6 (P <.001). Because of our linkage of problems and quality metrics in the EHR prototype, all users completed all metrics (21/21 metrics) (P <.001). Of note, at no point in the final usability evaluation did a clinician ask for data that was not in the system.

Figure 6

System Usability Scale (SUS) scores for the installed and prototype electronic health record (EHR). The mean score from our initial testing of the installed EHR was 47.1 (n = 39). SUS scores for the installed EHR did not significantly improve over the ensuing 3 years, with follow-up SUS score of 48.1 (n = 25). The prototype EHR demonstrated a substantial improvement in SUS score of 77.8 compared with the installed EHR used by the clinician (P <.001).

A tour of our prototype is available in Supplemental Video 1.

Discussion

The intent of the HITECH Act was to maximize the “meaningful use” of health information technology, specifically the EHR. A decade later, that goal has yet to be realized. The EHR is still viewed as a barrier to good patient care41, 42, 43, 44, 45, 46 and a source of clinician burnout.47, 48, 49 Our analysis confirms these concerns. In the 2–4 years between separate determinations of SUS scores of installed EHR systems, we observed no improvement. EHR functionality was still considered “poor.” In contrast, the EHR prototype scored 77.8. This represents a highly significant 30-point improvement in SUS scores, moving the EHR SUS score from poor to good, bordering on excellent. The salient question is “Why?” Why did this prototype achieve significant improvements in EHR usability when after billions of dollars have been spent on widespread EHR implementation is the EHR identified as a barrier to patient care and a major driver of clinician burnout?

We believe the answer comes from the cumulative approach embodied in this work—the compilation of multiple observations and interventions, where the whole becomes greater than the sum of the parts.

First is the methodology we used. Although simulated patients and complex clinical scenarios are common in clinician education, this method has been of only limited use in EHR usability studies. The use of simulation allowed us to gather observations across individuals and multiple different EHRs. This facilitated the identification of variations in practice but, more importantly, the identification of common themes and tasks. Additionally, the use of convergent parallel mixed methods coupled with agile development and action research, although not standard in most cardiovascular research, is the gold standard for human factors (human-computer interaction) research.

The second innovation was the ability to deconstruct and reconstruct the clinician-patient encounter tabula rasa. We adopted an actionable definition of usability (optimizing workflow and dataflow while reducing cognitive load imposed by the EHR) to focus our work on specific solutions. The ability to deconstruct the clinical encounter and then reconstruct the patient encounter with the input of independent clinical and informatics experts completed the arc of desired functionality. This tabula rasa approach allowed our clinical and informatics experts to develop a deep understanding of the wants and needs of clinicians independent of the constraints of legacy EHR systems and provided us with the freedom to validate our ideas through a fully functional prototype.

Few of the individual elements described here are completely novel. Position papers of the ONC, American College of Physicians (ACP), and American Medical Informatics Association (AMIA), as well as the writings of Carter,, Bodenheimer and Sinsky, Sinsky et al, and Melnick et al, provide excellent insight into the perceived EHR problems. This study validates many of the concepts and ideas laid out in those documents.

The functional EHR prototype described in this study satisfies the objectives laid out by the ACP, AMIA, and HL7 (Health Level 7) Reducing Clinician Burden Project, and, perhaps most importantly, demonstrates that a more useful and usable EHR is feasible. Understanding that dataflow needs were universal across sites allowed us to develop a framework to improve usability through the application of robust heuristics of good design.36, 37, 38 Promoting data and data persistence and incorporating patients into the health care team improved dataflow. The understanding of how expert clinicians collect and synthesize data prompted us to push relevant domain-specific content (including images) to clinicians. The use of 2 high-resolution screens and standardized functionality enabled clinicians to review, synthesize, and document with minimal need to open and close applications, thus reducing cognitive load (and physical action) imposed by the EHR. The implementation of a bookshelf metaphor allowed the clinician to separate the viewing of data from the necessity of including all viewed data in the clinical note. The linking of problems with problem-based connectors for diagnostic, therapeutic, and quality purposes improved clinician efficiency and effectiveness, and the ability to create custom views of the data for communication reduced extraneous, nonessential information, what is often referred to as “note bloat.”

Study limitations

This project is a proof of concept. Future work is needed to demonstrate that this framework is generalizable. Efficiency was not specifically measured; however, clinician comments were relied on for validation. The prototype did improve effectiveness (achieving 100% completion of PINNACLE quality metrics). However, a design assumption was that structured data were present and could pass from an inpatient to an ambulatory environment. Furthermore, effectiveness benefited because of the constructed quality documentation within the workflow and dataflow.

The final SUS evaluation of the installed EHR was based on clinician experience, not a direct comparison via simulated patient. Nonetheless, the conclusion still is valid owing to measurement of the overall experience of the clinical encounter rather than a specific design feature of the EHR. Furthermore, the similarity of SUS scores of initial and final installed EHRs and the large difference found with the prototype support this conclusion. Finally, self-reported measures of SUS by industry are substantially higher than what our subjects reported, although the mechanism of their recruitment is unknown. Bias was limited by excluding UNMC and ACC Taskforce members from the quantitative analysis, and the authors had no role in research subject recruitment. Finally, cognitive load was not formally measured, but SUS and single-value satisfaction scores were relied on as surrogates. Future usability research should include formal measurement of cognitive load.

Conclusion

Clinicians expect a well-designed, satisfying EHR that improves their efficiency and effectiveness. The prototype articulates the desiderata of functionality by clinicians and substantively validates that goal. The intent is not to create another EHR but to present a framework and roadmap to improve the usefulness and usability of all EHR systems. It is our intent that these findings stimulate a dialog among clinicians, informaticians, policymakers, and EHR vendors.

Acknowledgments

We want to acknowledge the contribution of Dr Mark Adkins for his facilitation of action research and the members of the American College of Cardiology Informatics and Health Information Technology Taskforce for serving as our expert sounding board.

Funding Sources

This work was supported by the Grant HS022110-01A1.

Disclosures

The authors have no competing interests to declare.

Authorship

All authors attest they meet the current ICMJE criteria for authorship.

Ethical Statement

The study was approved by the University of Nebraska Institutional Review Board (IRB). Individual site IRB approval was obtained when requested. Privacy and security concerns at each site were addressed prior to testing. Research subject selection was at the discretion of the individual study site. The research team had no role in subject recruitment. This research report conformed to the Helsinki Declaration guidelines as revised in 2013.