The development and electronic delivery of case-based learning using a fast healthcare interoperability resource system.

HL7 International's Fast Healthcare Interoperability Resources (FHIR) standard provides a common format for sharing health data (eg, FHIR resources) and a RESTful Application Programming Interface (eg, FHIR API) for accessing those resources via a FHIR server connected to an electronic health record system or any other system storing clinical data. Substitutable Medical Applications and Reusable Technologies (SMART) leverages FHIR to create an electronic health record (EHR) agnostic app platform. It utilizes the OAuth standard to provide for authorization and authentication. This paper describes the development and informal evaluation of Case Based Learning on FHIR (CBL on FHIR), a prototype EHR-connected FHIR/SMART platform to provide interactive digital cases for use in medical education. The project goals were to provide a more interactive form of CBL than is possible on paper to more realistically simulate clinical decision making and to expose medical students to modern informatics systems and tools for use in patient care.

BACKGROUND

Case-based learning (CBL; The terms problem-based learning [PBL] and CBL are used somewhat interchangeably to describe educational paradigms in which students learn by solving real world problems. In this paper we will use the terms CBL and PBL somewhat interchangeably with the common definition that they use realistic patient care situations to ask students to draw from their established foundation and make decisions about problems they may encounter in clinical practice. CBL may have been introduced to medicine in 1912 by James Lorrain Smith, the first full-time pathology professor at the University of Edinburgh in what he called the “case method of teaching pathology.” Students were asked to correlate the clinical history of patients, including their symptoms and signs, with the findings at post mortem by researching the patients’ cases from their clinical records. In the 1960’s Howard Barrows and others are credited with first introducing PBL to undergraduate medical education at McMaster University’s new medical school.

A 2016 review of the CBL literature begins with: “Medical and health care-related education is currently changing. Since the advent of adult education, educators have realized that learners need to see the relevance and be actively engaged in the topic under study. Traditionally, students in health care went to lectures and then transitioned into patient care as a type of on-the-job training. Medical schools have realized the importance of including clinical work early and have termed the mixing of basic and clinical sciences as vertical integration. Other human health-related fields have also recognized the value of illustrating teaching points with actual cases or simulated cases.”

Thistlewaite et al state that “The goal of CBL is to prepare students for clinical practice, through the use of authentic clinical cases. It links theory to practice, through the application of knowledge to the cases, using inquiry-based learning methods.” Peine et al found that “self-directed learning outperformed direct instruction in the environment of a modern, hybrid medical curriculum.”

Today CBL or PBL is widely employed in medical education in the United States as illustrated in Figure 1 from the American Association of Medical Colleges (AAMC). Figure 2 (also from the AAMC) illustrates that the CBL method is employed across medical specialties.

Figure 1.

CBL and PBL are widely used among the 141 US medical schools. (C) American Association of Medical Colleges, Used with Permission.

Figure 2.

CBL is employed in the United States in virtually all medical specialties. (C) American Association of Medical Colleges, Used with Permission.

LIMITATIONS OF THE CURRENT APPROACHES TO CBL

However, despite its wide acceptance, a study of 22 years of outcome research on PBL concludes that “evidence does not provide unequivocal support for enhanced learning through PBL.” An extensive 1993 review of the PBL literature cautioned readers about weakness in the criteria used to evaluate PBL and in study design but concluded that PBL “is more nurturing and enjoyable; PBL graduates perform as well, and sometimes better, on clinical examinations and faculty evaluations; and they are more likely to enter family medicine. Further, faculty tend to enjoy teaching using PBL. However, PBL students in a few instances scored lower on basic sciences examinations and viewed themselves as less well prepared in the basic sciences than were their conventionally trained counterparts. PBL graduates tended to engage in backward reasoning rather than the forward reasoning experts engage in, and there appeared to be gaps in their cognitive knowledge base that could affect practice outcomes. The costs of PBL may slow its implementation in schools with class sizes larger than 100.”

A further reading of the literature suggests some substantive areas for possible improvement in the CBL/PBL methodology. Group discussions are almost always utilized in CBL but they may not equally involve all students. Chang notes that a “variety of personalities co-exist in a PBL setting. Some students enjoy answering questions they know the answer to whereas other students are more shy about responding, perhaps in fear of voicing an incorrect answer.”

Cases or problems are typically provided to medical students as a paper or digital document (eg, PDF). Students are expected to read the case in the sequence in which the information is presented and pause at designated points to discuss questions posed in the document. There are obvious limitations in the degree to which a paper-based case can accurately represent real-world patients and clinical decision making, so a more interactive approach might be more effective. For example, like real patients, a digital patient might include an extensive prior history of hospitalizations and clinic visits, something that space would generally not allow on paper. Also, with a digital case a student might be presented with a choice of antibiotics one or more of which the patient is allergic to. Students who select those antibiotics might be taken down a path of the case that deals with the management of the allergic reaction. Again, this type of “branching” is difficult to do on paper.

Barrows proposed a more interactive approach called the Problem-based Learning Module which he described as “a specialized written simulation in which students are free to follow any course of action they deem appropriate as a patient case unfolds as it occurred in real life. They can ask any question about history, physical examination and laboratory or diagnostic test they wish and compare their actions with what really happened.”

ELECTRONIC HEALTH RECORDS AS AN EDUCATIONAL TOOL

Electronic health record (EHR) systems are increasingly the designated repository for clinical data used to make decisions about investigative procedures in order to make diagnoses, select optimal treatments and follow patients’ progress. EHRs are also at the center of a rapidly evolving digital ecosystem for care that includes new technologies for patient engagement and coordinated care among teams of health professionals.

The potential for using EHRs in health professional education has been previously recognized and explored in different contexts. Medical students in a 2010 pilot study of what the authors called “a student-centered EHR system for clinical education” identified a number of educational, practical and administrative advantages that the system offered over their existing ad-hoc procedures for recording patient encounters. The authors concluded that there is a “need to introduce and instruct students' (sic) on the use of EHR systems from their earliest clinical encounters, and to closely integrate learning activities based on the EHR system with established learning objectives.”

A group at the University of Michigan developed “VistA for Education” (VFE)—a customized version of the US Veteran’s Administration’s VistA Computerized Patient Record System—for use in health professional education. The University of Massachusetts Medical School developed the Academic Electronic Health Record (AEHR), “an experiential learning tool designed to help our students acquire and practice skills for clinical problem-solving and patient care though safe and effective EHR utilization of diverse and authentic patient cases.” The system operates within the Epic training environment and is described as being “integrated with classroom, lab and simulation learning the AEHR supports discipline-specific and interprofessional case-based learning in parallel with appropriate use of an EHR.”

Development of the Regenstrief Electronic Medical Record System (RMRS) began when health informatics pioneer, Dr. Clement McDonald, led an effort to develop a computerized patient management system for outpatient diabetes care. By 1999 it stored 1.5 million patients, and was used extensively by Eskenazi Health, a large essential health care system serving Indianapolis (formerly Wishard Memorial Hospital) and its more than 30 clinics. Over the past few years, with support from the American Medical Association, RMRS became the basis for the Regenstrief EHR Clinical Learning Platform. Students can use RMRS to virtually care for an extensive database of real, pseudonymized (real personal identifiers are either (1) redacted when it does not interfere with educational objectives or (2) replaced with realistic substitutes to keep documents readable) patients with multiple, complex health conditions by navigating records, documenting encounters, and placing orders within an application that is similar to EHRs used in practice.

These and similar efforts depend on a specific EHR environment tailored for educational purposes. We explored the use of HL7 International’s Fast Healthcare Interoperability Resources (FHIR) standard, a modern digital technology that is increasingly being used in clinical practice, to create a more interactive approach to CBL. The use of FHIR introduces the future possibility of connecting digital cases to FHIR-compliant EHR systems (with some necessary adaption to the specifics of each EHR’s implementation of FHIR). Currently, EHR systems have variable support for the creation of clinical resources using the FHIR Application Programming Interface (API), which is necessary for the simulated cases in our system. Because of this, at present we are using a simulated EHR with its own FHIR API.

Our goals were to:

Provide students with a more realistic simulation of actual medical practice by challenging them at each step in the diagnostic and treatment processes to make decisions about the next step in that process, while providing feedback as to why the selected step is not optimal or providing the information the correct next step reveals about the simulated patient.

Getting students to think about and discuss the potential role of informatics-based tools in clinical practice. Cases can be a logical point of departure to introduce informatics tools for purposes such as clinical decision support. Our solution allows the introduction of FHIR apps into digital cases by leveraging the increasingly accepted Substitutable Medical Applications and Reusable Technologies (SMART) standard for EHR connected health apps—many of which are intended for purposes that are potentially important in the management of patients similar to the simulated patient. FHIR apps designed for use in CBL would ideally be designed to teach students how to perform a clinical function (eg, such as cancer staging) in addition to helping them do it more easily and quickly. The use of FHIR could lead to an “ecosystem” of app developers with sharing among schools and possibly even the use of appropriate commercial FHIR apps in student education.

Develop our platform using a technology that is EHR agnostic so it could be the basis for an open community using informatics in support of CBL/PBL. Within such a community both cases and educationally oriented FHIR apps could be shared among institutions.

FHIR and SMART

FHIR introduces a common format for packaging health data. The format is instantiated in a family of “objects” (called resources) most commonly represented in Javascript Object Notation (JSON), a special text format commonly used for similar purposes in modern information systems, including the Internet. FHIR employs another commonly used technology, a RESTful API, to access resources. EHRs that adhere to the FHIR standard can present clinical data as FHIR Resources in response to a FHIR REST API.

Boston Children’s Hospital Computational Health Informatics Program and the Harvard Medical School Department for Biomedical Informatics recognized the potential to use FHIR as the basis for an EHR agnostic health app platform they call SMART and describe as “a health data layer that builds on the emerging FHIR API and resource definitions.” A FHIR application usually utilizes RESTful APIs to access data from an EHR through the EHR's server. At present most major EHR vendors provide FHIR server APIs. This increasing support of SMART means that software written to the FHIR and SMART standards can access and operate based on EHR data, no matter which underlying EHR a hospital or clinic is using (but often with some modifications to accommodate differences in vendor implementations of FHIR).

This technology has two key potential benefits for the development and support of an open EHR connected CBL/PBL environment:

FHIR and SMART are more approachable and facile than earlier software development technologies. This positively impacts the speed and cost of development and the ongoing maintenance of EHR connected tools such as our CBL on FHIR platform.

Cases and apps can operate in conjunction with most major EHR systems, so they are far more sharable across institutions.

CBL ON FHIR DEVELOPMENT

The prototype CBL/PBL platform was developed by a team of 10 undergraduate Information Technology students at The University of Queensland during a 2018 second semester special project course. We connected it to a simulated EHR environment developed by the CSIRO Australian e-Health Research Centre using the FHIR and SMART standards.

Three medical school faculty mentored the students and provided two sample cases for use in their work. Of these, a burns patient case was selected and the first two “triggers” (in CLB triggers are the material that is provided to serve as the basis for discussion of the case. A single case typically consists of several triggers) in that case were implemented in the prototype system.

The results achieved by the students who came to the course with essentially no prior knowledge of healthcare and no familiarity with FHIR and SMART support the approachability of the technology and the speed of development that is possible using it.

Students used the platform to convert data presented in the first two triggers into six appropriate patient, observation and medication FHIR Resources that accurately portrayed the simulated patient’s data (Figure 3). As illustrated in Figure 4, they stored these on the CSIRO FHIR server and the SMART app platform used OAuth to provide validation of the user’s identity and access privileges to the patient records.

Figure 3.

This small part of the Patient Resource illustrates how clinical data is stored in FHIR, documents that the patient is confirmed to be a smoker and codes that in SNOMED CT.

Figure 4.

A simple representation of the system architecture shows the role played by CSIRO’s FHIR Server (upper right) as the store for the patient record represented as FHIR Resources and in providing a SMART compatible app platform. Data flow would start with the Application requesting SMART OAuth authorization and then storing patient data as FHIR resources as it is revealed by student activity. In parallel student activities are stored for a planned future student activity reporting module.

As they proceeded through the case, the CBL on FHIR app challenged students to consider the next most appropriate care decision (eg, take further history, specific examinations of the patient, order specific investigations, and admit/discharge/transfer the patient) and presented possible options for the decision, as illustrated in Figure 5. Of these, some were intentionally incorrect or inappropriate at the stage of care of the patient presented in that trigger. The system provided students with the total count of correct actions and the number of those they had taken at any time point.

Figure 5.

Decision points within each trigger were presented as a menu of choices. Information obtained was stored in the simple EHR and could be reviewed at any point by selecting History, Physical or Investigations from the left menu.

Students were presented with the data that would have been obtained via the action they chose or were advised that the action was not an optimal choice at that stage of the case. The prototype system implemented a simple EHR-like user interface that presented the simulated patient’s record in sections labeled History (questioning the patient), Physical (examination of the patient) or Investigations (ordering laboratory tests, imaging studies, etc.). Any data obtained by the student was thereafter visible by selecting the appropriate section of the simulated EHR.

As in a real EHR, clinical data is exchanged as FHIR resources. This includes both the data that is made available to the student through the CBL tool (using Patient, Condition, Observation, and DiagnosticReport resources), and the clinical actions taken by the students (using Procedure, MedicationRequest, and MedicationAdministration resources).

One of our objectives was to expose students to the potential of informatics tools in clinical practice. The written version of the case explained that the patient’s Glasgow Coma Scale was 15. As illustrated in Figure 6, in the interactive CBL tool version of the case, the components of this scale are clearly illustrated to students as they document the clinical observations required to calculate the score. The Glasgow Coma Scale SMART on FHIR app uses the observations they record to perform the calculation.

Figure 6.

An example of a FHIR app developed for use in the burn patient case. It calculates the Glasgow Coma Scale using data obtained previously by examining the patient. The app is designed to make it clear how the score is calculated. Students can invoke it at any time by selecting “Glasgow Coma Scale” on the left menu.

PROJECT EVALUATION

To perform an informal evaluation of the app and how it might best be incorporated into teaching, it was deployed to three separate CBL tutorial groups of approximately 10 students each. All students evaluated the app using the System Usability Scale (SUS) questions for evaluating a wide variety of products and services, including hardware, software, mobile devices, websites, and applications. SUS provides a simple, reliable tool for measuring software usability. It consists of a 10-item Likert scale questionnaire with five response options from strongly agree to strongly disagree.

The first group gave the application a SUS score of 72 (62nd percentile of SUS scores). In this group there was a long period of discussion for each screen and students often lost track of where they were on the application.

The second group gave the application a SUS score of 84 (98th percentile of SUS scores). This group used the app continuously for 15 minutes before moving on to discussions. Students were also asked to provide written comments and this group particularly enjoyed the interactivity of the story and the accompanying applications.

The third group gave the application a SUS score of 53 (15th percentile of SUS scores). This group felt that the application was too “fragmented” and they would have appreciated better integration between the apps and the story. They felt the application hindered discussion and learning.

The tutors of both the second and third group expressed positive feedback regarding the capacity of this model to accurately simulate EHRs that students will use in their future everyday medical practice. Both tutors also cited reservations that the model could potentially hinder group discussion if students were able to work through on their own devices, rather than moving through as a group, using only one central screen. The tutor of the first group was involved in the app design process.

DISCUSSION

We believe that the variation in SUS scores is multifactorial. First, the mentors (the groups’ regular CBL tutors) had only a small briefing session in the use of the EHR-like system and FHIR apps just before the CBL session started. Improved training of the mentors will increase confidence and engagement with the system. Second, this pilot was introduced late in the semester into CBL groups with well-established group processes and group dynamics. Group dynamics can differ substantially between CBL groups, and the introduction of any new platform that guides group discussion and learning may thereby have different effects in different groups.

FUTURE WORK

Based on this prototype development and informal evaluation we have begun development of a more robust version of the platform that will include three modules: (1) a case authoring tool designed for use by clinical staff to construct complete digital interactive cases; (2) a “player” to present the case to students, and (3) a mentor dashboard to illustrate individual student and group performance as a guide to further discussion. We will use the new platform in a follow-on course for University of Queensland Information Technology students in the fall/winter of 2019. These students will develop FHIR apps to support educational opportunities identified by the initial digital case authors. We will conduct a more rigorous evaluation with the long term goal of offering the platform to other institutions.

FUNDING

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

AUTHOR CONTRIBUTIONS

Mark Braunstein and Jim Steel conducted the course in which the software was developed and were the primary authors of the paper. David Hansen was involved in structuring the course and reviewed the paper. Ben Barry, Sharon Darlington, and Iulia Oancea provided expertise relative to case-based learning and the specific case, reviewed the software design, assisted in the course and the evaluation of the prototype and contributed to drafting and review of the paper. James Battock, Daniel Cheung, Gregory Gan, Ben Hooper, Reilly Lundin, Duncan Nicol, Joshua O’Brien, Scott Whittington, Chris Wilkinson, and Tse Tse Wong developed the case-based learning app, and designed and oversaw the in-class evaluation, in consultation with the UQ Medical School and CSIRO staff, and reviewed the paper.