Current Perspectives on Augmented Reality in Medical Education: Applications, Affordances and Limitations.

This systematic review has been developed against a background of rapid developments in augmented reality (AR) technology and its application in medical education. The objectives are to provide a critical synthesis of current trends in the field and to highlight areas for further research. The data sources used for the study were the PubMed, Web of Science and Discover databases. Sources included in the study comprised peer reviewed journal articles published between 2015 and 2020. Inclusion criteria included empirical research findings related to learning outcomes and the populations for the selected studies were medical students. Studies were appraised in terms of to what extent the use of AR contributed to learning gains in knowledge and/or skill. Twenty-one studies were included in the analysis, and the dates of these suggested an increasing trend of publications in this area. The uses of AR in each selected study were analyzed through a lens of affordance, to identify which specific affordances of AR appear to be most effective in this domain. Results of the study indicated that AR seems to be more effective in supporting skill development rather than knowledge gain when compared to other techniques. Some key affordances of AR in medical education are identified as developing practical skills in a spatial context, device portability across locations and situated learning in context. It is suggested that a focus on relevant affordances when designing AR systems for medical education may lead to better learning outcomes. It is noted that the majority of AR systems reported in the selected studies are concentrated in the areas of anatomy and surgery, but that are also other areas of practice being explored, and these may provide opportunities for new types of AR learning systems to be developed for medical education.

Introduction

Augmented Reality (AR) is a technology that allows for virtual (digital) context to be overlaid on the real world, using hardware such as headsets, smart glasses, or mobile devices. These overlays can be related to the real-world context using physical markers (such as QR codes) or can be “markerless”, with other technologies such as sensors being used to identify position, locations, or objects. In the continuum between real and virtual environments, AR resides towards the “real environment” end of the spectrum.1 This link to the environment enables AR to be a mobile technology, with the user able to navigate in physical spaces during AR experiences. AR allows for the embedding of digital, location-specific, and contextual information into a physical space and thus enables learning to be enhanced and contextualized.2 There are several potential advantages to maintaining a link with the surrounding physical world when learning, which is what differentiates AR from Virtual Reality (VR). These include the safety aspect of having the freedom of sight needed to move around the learning space,3 allowing visual teacher and student interaction,4 and supporting learning within the context of a physical location.2 In the context of medical education, AR offers some specific advantages, such as being able to be deployed in a professional work environment, simulating relevant aspects of real-world tasks, giving immediate learner feedback and not always requiring an expert or instructor to observe trainee performance.5 One feature of AR in medicine, it is interesting to note, is that in some areas it has already become embedded into everyday practice. For example, the ProMIS augmented reality laparoscopic simulator was used in the assessment of a virtual reality simulator, with the AR tool being already established as a valid means of measuring medical skill.6 This example indicates that AR has already gained acceptance in medical education. However, systems such as ProMIS are from a previous generation of AR in medical education and a systematic review of more recent approaches is called for.

Study Objective

The objective of this study was to provide a comprehensive literature review of the uses of AR in medical education that have been empirically examined. The application of AR in the context of medical education is an area of research that has been increasingly investigated in recent years.7–16 There have been several literature review articles published that have addressed various aspects of how AR has been used for medical education, frequently focusing on areas of specialization. For example, systematic reviews in medical specializations have recently been undertaken in surgical education,7,8 anatomy,9,10 radiology,11 and neuronavigation.12 A further review of research into AR in orthopedic surgery also included some examples of education and training.13 There have also been some more general analyses of the literature pertaining to AR in medical education, for example a review of articles relating to AR for healthcare education in areas other than surgery,14 while Barsom et al and Tang et al have produced systematic reviews of augmented reality across all aspects of medical education.15,16 Both these studies focused specifically on the different types of AR applications (ARAs) adopted in medical education. Barsom et al’s study, which included literature up to 2015, identified only seven applications in this context. In contrast, Tang et al identified 16, with literature explored up to 2018. Only two systems (ProMIS and Immersive Touch) appeared in both reviews, suggesting that AR technology in medical education is both evolving and expanding. This background suggests that a systematic review of the recent literature that addresses AR across medical education is warranted. It further suggests that a focus away from the specifics of particular AR applications is needed. Rather, a more holistic and encompassing view of the AR learning experience would be a useful contribution to the literature. In this context, this review addresses literature published between 2015 and 2020 (inclusive) and adopts an affordance view in analyzing the selected literature. It therefore offers an updated view of AR in medical education that is not application focused and offers new perspectives on this context by applying an affordance lens to the analysis (see next section), therefore supporting a deeper exploration of what makes AR especially effective for medical education and informing further development and engagement within this area.

Affordances

In this study, we have chosen to examine the selected studies through a lens of the affordances of AR. The value of analyzing affordances is that they help examine user goals, and thus the motives driving the use of technology. Affordances are also relatively generalizable and constant across specific implementations so are not tied to particular ARAs. They also give us a means of comparison when evaluating AR against physical reality, such as using a wet lab39 or organ cross-section.18 Importantly, there is no single set of affordances for AR in education, since they are highly contextual. For example, the users’ perceived affordances of an AR system for language learning while visiting a city overseas are very different to those perceived by a medical student who needs to develop a particular clinical skill.

Steffen et al identified four primary affordances that they applied to both augmented and virtual reality outside the context of education: 1) To diminish negative aspects of the physical world (eg, to reduce physical risk), 2) to enhance positive aspects of the physical world (eg, to facilitate additional information), 3) to recreate existing aspects of the physical world (eg, to reduce resource costs), and 4) to create aspects that do not exist in the physical world (eg, by appearing to operate outside the normal laws of physics). They also identified two modifiers that are important for AR: sensory vividness and physical context.18 However, these categories have been developed specifically in the context of comparing VR and AR, so although we believe they are useful in identifying the affordances of AR when compared to VR (as some of the studies in the literature review have done) they cannot be adopted for other contexts uncritically.

In the context of education, Bacca et al identified three affordances of AR: 1) Blending two environments (digital and real) to place learning within a physical learning environment and make learning more concrete and situated, 2) integrating a range of digital artefacts that can be created, shared and collaboratively explored, and 3) engaging with 3D digital objects from multiple perspectives.19 Also in the context of education, MacCallum & Jamieson outlined four affordances of AR: 1) visualization of the 3D and the invisible, 2) contextualized information, 3) portability of the device to interact with the location, and 4) social and shared engagement.20 These perspectives, of course, place more emphasis on contributions to learning, for example those that support collaboration, engagement and creativity, but also offer a more nuanced view of the detail of the visualization, the AR technology and the context. They also explore these features within the general context of education, and it should be noted that some affordances may be more dominant in some domains than in others.

In the more focused domain of science learning, Cheng & Tsai identified spatial ability, practical skills and conceptual understanding as affordances related to visualization, and scientific inquiry learning as being context-based.2 Similarly, in the context of medical education, a literature review carried out by Sen et al identified only two affordances: 1) facilitation of context-aware real-life situational learning, and 2) 3D visualization.21 It is notable that the two contexts of visualization and situated learning come to the fore when examining scientific and medical learning.

Across these studies of affordance that address related domains of interest, we can identify the following as being of interest within the specific constraints of this literature review, ie, affordances of AR that can enhance learning in medical education (codes added for later reference):

A1: Reducing negative impact (risk, cost).18

A2: Visualizing the otherwise invisible.20,21

A3: Developing practical skills in a spatial context.2

A4: Device portability across locations.20

A5: Situated learning in context.18–21

Method

The PICO framework is frequently used for literature search strategies in medical contexts. Although this study is confined to medical education and not to patient outcomes, we have used elements of this framework in structuring our analysis, namely the population, the intervention, the comparison and the (learning) outcomes. In addition, some authors add timing and study type to the PICO framework. In this study, our timing is the period of publication and we examine the study design in categorizing the literature we have selected. Table 1

Table 1

Characteristics of the Selected Studies

	Author	Year	Specialty	Study Design	Population	Effective Measure	Affordance*	Approach
1.	Abhari et al23	2015	Neuroanatomy (Tumour resection)	Participants simulated the planning of a tumour resection performing 3 tasks in 4 environments over 8 repeated trials	11 graduate students and 11 experts. No experimental or control groups.	Task performance and speed both improved when compared with traditional approaches	A2, A3, A5	Vuzix mixed AR/VR headset used with a stylus and a head phantom.
2.	Aebersold et al24	2018	Nursing (NGT insertion)	Experimental and control groups tested on their ability to place a nasogastric tube (NGT), assessed using a 17-item competency checklist.	69 Nursing students (35 experimental, 34 control)	The experimental group performed significantly better than the control group on their ability to correctly place the NGT	A2	Experimental group used an iPad-based AR training module
3.	Andersen et al25	2016	Telementoring	Participants completed port placement and abdominal incision tasks, assessed on 3 metrics	17 pre-medical and 3 medical students (10 experimental, 10 control)	The experimental group completed tasks with less placement error and with fewer focus shifts but more slowly.	A1, A3, A4, A5	Experimental group used STAR (System for Telementoring with Augmented Reality) based on a tablet with live annotations. Control group used a separate monitor.
4.	Barmaki et al26	2019	Anatomy	Four-group Solomon design with two control and treatment groups, - one group pre and post, the other with just post-test. Knowledge assessed by pre- and post-tests, body painting accuracy assessed by experts	288 undergraduate premedical students (164 experimental, 124 control)	The system enhanced learning of the musculoskeletal system with improved knowledge retention and increased time on task	A3, A5	Used the REFLECT (augmented REality For LEarning Clinical anaTomy) magic mirror (MM) system using Microsoft Kinekt (experimental) where the organs were shown in situ or virtual mirror where organs are identified in the textbook (control) where students applied anatomical body painting.
5.	Bogomolova et al27	2020	Anatomy (lower limb)	Double-center randomized controlled trial. Visual-spatial abilities and anatomical knowledge were assessed	59 biomedical undergraduates (20 experimental AR, 20 experimental 3D, 19 control)	There were no significant differences on outcomes, but the AR tool showed particular value for those with lower visuospatial abilities	A2, A3	Compared AR model using Hololens with 3D model and anatomical atlas.
6.	Bork et al28	2017	Anatomy	AR MM used for anatomy learning. Participants were asked to identify the correct placement of virtual organs in four different views to compare the impact of reversing MM	20 medical students - all students experienced all conditions (pre-test and tool-assisted test)	Students in general identified correct organ locations significantly better and faster in NRMM conditions compared to RMM conditions	A2, A3, A5	Reversing (RMM) and non-reversing magic mirrors(NRMM) compared - organ positions also flipped
7.	Bork et al29	2019	Anatomy (radiology)	A pre- and posttest design with multiple choice questions used to assess learning using radiology cross sections across three conditions	72 students (24 experimental AR, 24 experimental Anatomage, 24 control)	The MM system achieved significant improvements of scores. Also showed a greater benefit for students with low mental rotation	A1, A2, A3, A5	AR MM system compared to the Anatomage, a virtual dissection table or traditional radiology atlases
8.	Ebner et al30	2019	Ultrasound (kidney)	Prospective 2-armed study. Participants measured on accuracy and time taken for kidney measurements using an ultrasound machine after the training period	66 medical students (33 experimental, 33 control)	Use of the mobile app for training purposes improved the quality of ultrasound kidney measurements. There were larger, more realistic values in the study group and measures were all valid in the study group but not in the control group	A1	Both groups used textbooks as preparation; in addition, the study group had access to a mobile AR ultrasound simulation app.
9.	Gierwiało et al31	2019	Surgery (liver syringe insertions)	Participants simulated a liver biopsy and thermoablation in two conditions. Accuracy of the needle position was assessed	25 medical students and 3 surgeons	The ratio of failed syringe insertions was reduced from 50% to 30% by using the AR tool.	A1, A2, A3, A5	The MARVIS AR system, projecting onto a 3D liver phantom, was compared to a 3D model on a monitor
10.	Henssen et al32	2020	Neuroanatomy	Pre and post-tests on neuroanatomy were administered to both groups	31 Medical and biomedical students (15 experimental, 16 control)	Cross-section group showed significantly more improvement on test scores than AR students. the cross-section group, experienced a significantly higher cognitive load		Experimental group used the GreyMapp-AR tool for visualization of the brain, while the control group used cross-sections
11.	Koutitas et al33	2020	EMS First Responders (locating correct equipment)	Performance measures of speed and accuracy of locating 10 items on an ambulance bus	30 cadets (no figures provided for experimental AR, experimental VR and control, but assume 10 each)	AR performed better than the control group but less well than the VR group.	A3, A4	The AR group used a Hololens, the VR group used an Oculus Rift and the control group received the currently recommended training
12.	Küçük et al34	2016	Neuroanatomy	Comparison of learning outcomes from neuroanatomy topics, assessed through a pre- and post-test using multiple choice questions	70 medical students (34 experimental, 36 control)	The experimental group, which used mobile AR (mAR) applications, reported higher achievement and lower cognitive load.		Experimental group used a MagicBook mAR tool (created using Arasma) control group used traditional presentation materials (including 2D pictures, graphs and text)
13.	Logishetty et al35	2019	Surgery (hip replacement)	Participants introduced to six clinically relevant cup orientations for positioning a hemispheric acetabular cup. Assessed on performing each orientation with the orientation error° measured using a head-mounted tracker camera.	24 medical students (12 experimental, 12 control)	No difference in error between participants trained to orient the cup implant by AR or by an expert surgeon after a structured training and assessment program. Students saw AR as a tool for unsupervised training, to supplement learning with an expert surgeon inside the operating room.	A1, A2, A5	Experimental group used an AR headset (Hololens) and a simulated Total hip arthroplasty. (THA). Control group received one-on-one training from a hip arthroplasty surgeon
14,	Moro et al36	2017	Anatomy	Students completed a lesson on skull anatomy, after which they completed an anatomical knowledge assessment.	59 medical students (20 experimental VR, 17 experimental AR, 22 control).	No significant differences were found between mean assessment scores in VR, AR, or TB. R participants were more likely to exhibit adverse effects such as headaches dizziness or blurred vision		3D models provided in three different modes: AR, VR and tablet-based (TB). VR delivered using Oculus Rift, same basic app written in JavaScript and Unity and deployed in all three modes
15.	Mu et al37	2020	Surgery (Percutaneous renal access - PCA)	Skills assessed before and after training with the AE simulator. Seven metrics recorded by the simulator to evaluate user performance.	24 postgraduate medical students, 6 experts (pre and post-tests no experimental or control groups)	Performance improvements of the students in both objective and subjective evaluation after training	A1, A2, A3	The AR simulator allows the user to practice PCA on a silicone phantom using a tracked needle and ultrasound probe emulator under the guidance of simulated ultrasound on a tablet.
16.	Noll et al38	2017	Dermatology	Dermatological knowledge was ascertained using a single choice (SC) test (10 questions) as a pre and post-test, and a follow up test for retention	44 medical students (22 experimental, 22 control)	For the single choice tests, there were no significant differences in learning. in the follow-up test after 14 days, experimental group had retained more knowledge	A2, A5	The mobile AR App mARble-dermatology provided content using digital flashcards. AR content was linked to paper-based markers placed on the skin of users. Each group had the same app but control group did not have the AR markers
17.	Peden et al39	2016	Surgery (suturing)	Students with no prior suturing experience undertook a practical assessment, where suturing was videoed and graded by masked assessors using a 10-point surgical skill score	14 medical students (4 experimental AR assisted teaching, 5 experimental AR self-learning, 5 control)	Suturing ability after teaching was similar between groups. No difference in number of sutures placed between groups	A1, A2, A3, A5	head-mounted displays for surgical training compared with conventional wet-lab teaching methods.
18.	Rai et al40	2017	Ophthalmoscopy	Evaluations were completed on the simulator, with 3 tasks, and outcome measures were total raw score, total time elapsed, and performance.	28 postgraduate medical students (13 experimental, 15 control)	The AR group performed better than the control group on all 3 outcome measures. However, the simulator cannot completely substitute real patient examination experience.	A1, A2, A5	Compared the impact traditional teaching approach of binocular indirect ophthalmoscopy (BIO) to the EyeSI AR BIO simulator for eye examination compared to just training on the system
19.	Rochlen et al41	2017	Surgery (central venous catheterization)	Participants were required to place a central venous catheterization needle in a mannequin. Two trained raters independently documented performance via an assessment checklist	40 subjects, including 20 medical students + anesthesiology residents and faculty. No control group	Needle placement ability was similar between experienced and non-experienced participants but the less experienced were more likely to inadvertently puncture the carotid artery	A1, A2, A3, A5	AR glasses were provided to project internal anatomical landmarks, Participants were then asked to place the needle without the benefit of the AR
20.	Sugand et al42	2019	Surgery (dynamic hip screw (DHS) guide-wire insertion)	Randomized controlled trial with five real-time objective performance metrics.	45 medical students (23 experimental, 22 control)	A significant difference between groups was demonstrated as the training cohort significantly outperformed the control cohort in three metrics	A1, A2, A3	The AR tool was FluoroSim, an interactive fluorescence microscope simulator
21.	Wang et al43	2020	Neuroanatomy	Text and images from two clinical neuroanatomy textbooks were deployed into the three conditions. Neuroanatomy learning on the visual pathway was assessed for retention.	52 second-year medical students (19 experimental AR, 15 experimental 3DM, 18 control)	The AR group demonstrated higher retention in both the nominal and spatial type information for at least a month compared to the other groups.		Three learning tools assessed: AR using Hololens 3D visualization on a 2D screen (3DM), or text-only

Notes: *A1: Reducing negative impact (risk, cost), A2: Visualizing the otherwise invisible, A3: Developing practical skills in a spatial context, A4: Device portability across locations, A5: Situated learning in context.

The main question being addressed in this review is: “what evidence can we find in the recent literature for the effectiveness of augmented reality in medical education?”

In order to provide a contextual frame for the analysis, a sub question is “what affordances are evident in the selected literature that can support learners in a medical education context?”

Eligibility Criteria

Sources included in the study comprised peer reviewed journal articles published in English between 2015 and 2020. Eligibility criteria included empirical research findings related to learning outcomes. The populations for included studies were medical students. We included students who were studying nursing or paramedicine in this category where the studies were medical in nature. We also included one article where the population were only described as “non-clinicians”, since they appeared to be in learning roles in the study. We excluded studies where the population were described as “residents”. Not including this population might be considered as a limitation of this study, and this is addressed in the conclusion.

Studies excluded from this review were those that explored the uses of AR in medical practice rather than in formal education contexts, and work that focused on the design of AR tools for medical education and did not include an empirical evaluation of medical students’ learning outcomes. We also excluded veterinary and dental practice, health and safety contexts and any studies where the population were school students rather than medical students. We did not include any studies where the data gathered was purely qualitative or measured only perceived aspects such as enjoyment or engagement, though some of the selected studies included this type of evaluation alongside empirical assessment of learning outcomes.

We used PRISMA (Preferred reporting items for systematic reviews and meta-analyses) to structure this systematic review.17 It should be noted, however, that the subject of this review is not clinical outcomes but learning outcomes, and these are subject to many uncontrolled variables. The focus of this review is to assess the affordances of AR that have been leveraged in the articles selected and to assess how they might relate to the outcomes of each of the reported studies. Therefore, we have not included separate qualitative synthesis and quantitative meta-analysis in our process. Rather, we have focused on the relationships between AR affordances and learning outcomes in different contexts.

Information Sources

Three databases were used for the literature search:

PubMed

Web of Science: Including Web of Science Core Collection‎, Biological Abstracts‎, FSTA (food science resource), MEDLINE‎, KCI-Korean Journal Database, SciELO Citation Index, Current Contents Connect‎, Centre for Agriculture and Bioscience International‎ and the Russian Science Citation Index.

EBSCO Discover: Including MEDLINE, Academic Search Premier, Complementary Index, Scopus and the Science Citation Index.

The search terms used in the databases were “augmented reality”, combined with “health” or “medical”, and “learning”, “education” or “teaching”. The date range of publications searched was 2015–2020 inclusive, and searches were filtered to return full text, peer reviewed journal articles only. These search terms were tested along with some alternatives against the Web of Science database. For example, using “virtual reality” as an exclusion criterion removed some relevant articles, because these are sometimes compared in the same studies, so such exclusions had to be done manually.

In the initial search, 556 articles were returned from the Web of Science, 147 from PubMed and 898 from EBSCO Discover, a total of 1603. There were many duplicates both within and across these aggregated databases, with some articles appearing multiple times. After duplicates were removed, the remaining 875 articles were checked by both authors for potential relevance by reading the titles and abstracts and then confirming that full texts were available for all items returned by the searches. Articles were removed if they were about virtual reality, 3D modelling or visualization, video games, school education, or health and safety training. Articles were also removed if it was clear that they only mentioned AR in passing, were speculative with no empirical content, were about technology but not education, related to the learning of patients rather than practitioners, or were about patient diagnostics in practice rather than in education. At the end of this screening process, 97 articles remained.

Study Selection

The screened results were exported as Endnote files and then imported into Rayyan, a web application designed to support the process of systematic literature reviews.22 The two authors then independently examined the full text of all the remaining articles against the inclusion criteria using the blind review tool included in Rayyan, during which both reviewers screened the candidate articles against the inclusion criteria. After the blind review stage, 20 articles had been included by both reviewers and 64 had been excluded (84 out of 97). There were a further 4 conflicts, and another 9 articles where one of the reviewers had classified an article as a “maybe”. In only two of these had the other reviewer excluded the article. The inter-rater agreement (based on percent agreement) was 86.5%. If paired combinations of “maybe” and “include” are included, inter-rater reliability was 93.8%

After further investigation of articles where there was no definitive agreement, three of the four conflicts led to rejection of the paper. One was included once the suitability of the population had been clarified. Only three of the nine “maybes” were included, following further analysis of populations, study types and practice contexts. At the end of this process, there were 24 articles to be included in the final stage of the review. While these 24 articles were being critically examined in detail, three more were excluded. One because it appeared that none of the empirical results directly related to the measured attainment of medical knowledge or skills, and another because although it used “augmented” terminology and referred to 3D models and augmenting the learning, it was not clear if any AR technology had been used in the study. A further article was excluded because it used an AR system to compare a VR system, but the AR system itself was not evaluated. This reduced the final number of articles in the review to 21.

Figure 1 shows the numbers of studies screened, assessed for eligibility, and included in the review using a PRISMA flow diagram.

Figure 1

PRISMA flow diagram of the screening process. Notes: PRISMA figure adapted from Liberati A, Altman D, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Journal of clinical epidemiology. 2009;62(10). Creative Commons.17

Study Characteristics

Table 1 shows the characteristics of each of the selected studies. For each article, the table provides the author, the date (year of publication), the medical specialty addressed, the study design, the population, the measures of effectiveness, the related affordances and the approach (technologies and other materials). The populations are described in terms of the number of participants in the experimental and control groups (where applicable). Although studies selected for this review were chosen because they included medical students, some studies also included experts in the population. The relevant affordances are referenced using the five codes previously defined.

Discussion

The selected studies explore a range of ways in which AR tools can assist in the education of medical professionals. The extent to which digital tools can substitute for, or enhance, the learning of medical knowledge and skills depends on many factors. AR is often used as one part of the learning experience where its adoption is usually driven by the special qualities it brings to the learning process. It is these properties that will support its further adoption as a valuable tool for medical education. In addition, AR is often used as one part of a larger simulation system that allows for safe development and practice of both practical and diagnostic skills. In this context, if there are limitations, then these are outweighed by the benefits of being able to train without patients or cadavers. For example, the EyeSI BIO AR simulator is unable to simulate scleral depression so cannot completely substitute real patient examination experience, but training in binocular indirect ophthalmoscopy, a foundational skill in ophthalmology, can be tedious and challenging. Thus, the AR simulator serves a valuable role despite its limitations.40 Therefore, while AR may not be able to substitute for all learning activities there are some contexts where AR is uniquely positioned to provide an enhanced learning experience. For example, “many mannequin-based trainers are limited by the inability of the trainee to view the internal anatomical structures” (p.57).41 AR tools are particularly strong in the area of allowing medical students to “see inside” the body, so one area where AR has been commonly incorporated into medical education is in the teaching of anatomy. Previous review articles have tended to focus on the application level of AR. By exploring these studies through an affordance lens, we can move the discussion away from tool or domain level discussions to examine which AR affordances can best support learning outcomes. However, before we consider this aspect we will briefly explore other categories of analysis in order to frame the research.

Categories of Study

Table 1 includes the medical specialty of each of the selected articles. As noted previously, anatomy (in particular, neuroanatomy) is a popular area of investigation for AR studies, with nine of the selected articles relating to this area. This may be because of the financial, ethical, and supervisory constraints on the use of cadavers36 as well difficulties of sourcing sufficient numbers for anatomy training, but there are also the additional insights that augmented reality visualizations can provide to anatomy students.34

The other popular area of investigation for the selected articles is surgery, with six studies in this area. A common use of AR in these studies is in conjunction with physical phantoms (such as silicon models of human organs)30,37 and other simulation equipment that enables medical students to practice activities such as inserting needles into patients.31 These types of studies seem overall to show more success than those where AR is just a visualization that is not linked to another physical artefact. This emphasizes the importance of the reality-virtuality bridge that augmented reality can provide.

The prevalence of these areas of focus is not unexpected. Kamphuis et al’s examination of AR in medical education highlighted visualizing human anatomical structures and surgery skills training as key areas of interest for applying AR.5 The third area they identified was visualizing complex systems of the human body, including dynamic simulation. Three of our articles could be seen as relating to this area; ultrasound,30 dermatology,38 and ophthalmoscopy.40 In addition, one article related to telementoring, though this also falls into the category of surgery,25 one to nursing,24 and another to paramedics correctly locating medical equipment.33

The studies included in our sample were all chosen because they included empirical results from the measurement of improved learning outcomes based on the use of AR. There are two areas of learning outcome that have been measured in these studies: the development of technical skill (10 studies) and the acquisition of medical knowledge (11 studies).

The areas of technical skill addressed in these studies include inserting a needle,31,41 a wire,42 or a tube,24 suturing,39 port placement,25 positioning a hemispheric acetabular cup,35 percutaneous renal access,37 tumor resection planning,23 and ophthalmoscopy.40 The main measure in these studies is accuracy, although in some studies the speed of completing the task is also considered.23,25

Most of the studies that addressed the acquisition of medical knowledge focus on physical anatomy26–29,36 or neuroanatomy.32,34,43 The other studies that measured knowledge acquisition were in the fields of dermatology,38 kidney ultrasounds,30 and paramedicine.33

The measurement of learning outcomes in these studies is based either on comparing experimental and control groups, sometimes addressing three different conditions,27,29,36,39,43 or a pre-test and post-test on a single group.23,31,37,41 As well as knowledge being examined immediately after the experiment, in some cases knowledge retention was also measured after a period of time.

In the studies, in our sample, where the AR experimental group did not outperform other groups,29,32,35 there is nevertheless the suggestion that AR can provide a useful supplement to other learning conditions. However, the AR group did show lower performance in their learning gains than the other condition in one study. We will address these outcomes in the discussion of affordances.

Another aspect of these studies is the comparison between AR used as a substitute for learning materials as opposed to being used to simulate physical actions in a training context. In other words, gaining knowledge through an AR tool does not (in our sample of the literature) tend to out-perform other modes of learning. However, when AR is used for practical aspects of medical education, such as developing accuracy and speed in diagnoses and procedures, this is where it appears to provide unique and additional benefits.

Other Aspects of Evaluation

Most studies included in this review focused solely on the quantitative assessment of the AR experience. However, some studies did assess aspects of user perceptions, using either open comments or Likert scale assessment, along with some other qualitative aspects of the use of AR.

The feedback from students was generally very positive, with most students reporting that the various experiences were enjoyable,24,33,36 effective for learning,24,26,27,29,32–34,36,41,43 fun29,32,41,43 and novel,24,32,41,43 and they preferred learning through AR over more traditional means.26,27,33 Though some studies found it was easy to use the AR systems,24,27,32 especially compared to VR,33,36 there were some negative issues highlighted by the students involved in these studies. These issues included problems with the design and usability of the AR, such as difficulty with using the lenses when wearing glasses41 and other issues like the performance of the system or limited features.29,41 In addition to these specific system-related issues, there were broader issues raised that were considered to limit the wider uptake of AR, especially with regard to the more complex systems. These included the cost of some of the tools used to support the AR experience, such as Google Glass,39 and Hololens33,35,43 as well as the overall cost of development39,41 along with general limitations of these systems such as short battery life and low recording quality.41

In general, only the studies that included VR referred to negative learner experiences when using the system. Moro, et al36 compared the use of VR, AR and 3D tablets, evaluating a number of symptoms of use, both eye-related and more general. They noted that VR use was associated with a greater degree of discomfort, headaches, dizziness, nausea, disorientation, blurred vision, difficulty focusing on images, and double vision. These effects were not reported by users of the AR version of the system. However, one symptom that was found across the board (VR, AR, and 3D tablet) was the difficulty students had in concentrating on the learning. The study found that students were often distracted by the technology, and as a result did not always focus on the learning tasks. A further issue was identified by Bork et al29 where the use of the system often resulted in higher student fatigue due the level of concentration required. This fatigue was also found in other studies34,36 where systems had a higher cognitive load and therefore required students to concentrate more than they would with other more traditional approaches.

The Role of Affordances

Drawn from the literature we identified five affordances that would be specific to medical education. Table 1 indicates where these play an important role in a given study. Affordance A1 (reducing negative impacts such as risk and cost) particularly applies in those studies where the alternative to using an AR tool would be working on live patients, which may carry risks for them. However, the more likely impact in these studies is in terms of cost, which may come from the costs of alternative equipment (VR for example is frequently more expensive) the costs of more training being required (in cases were AR improved learning outcomes) and the cost of alternative means of learning (such as the provision of cadavers or wet labs). For affordance A2 (visualizing the otherwise invisible), it might be assumed that this is applicable in all AR contexts. However, in the studies examined here, several of these evaluate AR in comparison with other tools, so that there are alternative ways of visualizing the invisible (for example VR). In these cases, the visualization is not a unique characteristic of AR in the experimental condition. However, the way that learning can be conceptualized in context, such as in the examples that use magic mirrors (MM), they provide a unique affordance over VR (A5). Affordance A3 (developing practical skills in a spatial context) applies to many of the studies that focus on working with physical materials (such as phantoms and mannequins) through an AR layer that adapts to the viewpoint of the learner. Notably, it does not apply in situations where the AR is only being used to render 3D virtual models based on a trigger such as a textbook. For affordance A4 (device portability across locations) the key aspect is how the device is able to draw in a location or augment the learning in a location. For example, in Koutitas et al’s33 study where AR is used to bring a location (the ambulance bus) into the learner’s environment. In Andersen et al25, AR was used to augment the learning environment, where annotations from the simulated mentor are provided to the students to support their placement of the incision. It is also important to clarify that this affordance goes further than just making the experience mobile. While mobile AR (mAR) enables learning to take place anywhere, therefore enabling learning to happen outside the classroom and lab, it is not necessarily that the learning is portable, as in the full sense of the affordance discussed in this study. While mAR supports learning outside of specialist rooms like dissection rooms, such as in Henssen et al32, where GreyMapp was used to visualize the 3D anatomy of the brain, the learning experience was not necessarily portable. While some examples of AR are not portable, for example the ophthalmoscopy simulator and MM systems, we are considering portability to be more than just the ability for the user to move the experience or position the device. Finally, affordance A5 (situated learning in context) applies where the physical context has real meaning for the learner. Therefore, while A4 draws on the idea of drawing in a location, A5 takes that next step of the location/object in that location being fundamentally drawn into the AR experience. Two examples where this is the case are the EMS First Responders scenario and telementoring. In these cases, the context is highly relevant. In other cases, for example where AR book-based learning material is being studied, the context has no impact.

Notably, there were four studies where none of the affordances that we had proposed as being specifically relevant to medical education were addressed. In these cases, the AR treatment was being used only as study material in situations where the same study material was available using other sources. Here, while features of AR were being utilized (ie, overlying of virtual material on a physical background), these features did not offer any of the affordances from the set we had identified from the literature as being potentially effective in this context. In three of these studies, the alternative materials were text or other visualizations such as 3D images and VR,34,36,43 while in the fourth case, the AR visualization material was substituting for physical brain cross sections that were available to the control group.32 In one of these studies, there was no significant difference in the learning outcomes of the AR experimental group.36 In the cross-section group, the control group outperformed the AR experimental group.32 One study claimed a significantly better learning outcome for students studying anatomy using an AR textbook compared to students using a standard textbook.34 Some qualitative student feedback indicated that the AR visualization had assisted their learning, so therefore we might consider this study as being borderline in terms of addressing A2 (visualizing the otherwise invisible). In the fourth study that did not address any of our chosen affordances, three learning tools were assessed AR, 3D visualization and text-only.43 In initial learning outcomes, the text-only group outperformed both of the others. Only in longer term retention did the AR group show better performance, and then mostly in the spatial rather than the nominal area of anatomy knowledge. It may also be pointed out that while the study exploring the use of 3D visualizations of the brain indicated that there was an assumption that this AR would decrease the need for dissection rooms (therefore could be considered as A1 by reducing costs), the comparison was on cross-sectional samples, which did not require these rooms either.

A further study that addressed only A224 raised some questions about whether it was, in fact, an AR system, even though it was described as such by the authors. Interactive models were used to augment video content, but there were no external triggers, nor links with the context. In conjunction with some of the articles that were excluded only on closer examination of the full text, this suggests that the terminology of AR needs to be used in a more careful way by researchers in this field.

Turning our attention to the different affordances addressed by various studies, a notable aspect of A1 is that the AR systems that provided a low-cost alternative to either systems or experts were able to effectively substitute for those alternatives.29,35,39,40 A2 (visualizing the otherwise invisible) plays an unsurprisingly strong role in studies that assist students to develop their physical skills by enhancing the learner’s view of a task with supplemental information.23–26,28–31,35,37,39,40,42 A3 (developing practical skills in a spatial context) frequently overlapped with A2 in our analysis (in 10 out of 21 studies) because of the importance of AR visualization in supporting skills development in medicine, where physical objects (patients, organs, etc.) are at the heart of knowledge. However, this affordance is not uniquely tied to visualization since some visualization is not about practical skills development. We also note that skills development may take place in the absence of visualization of objects. For example, visualization may be limited to directional cues.33 When exploring A4 (device portability across locations), while all mAR approaches have some aspects or potential of portability, it is rather the studies that draw on this specifically which really draw out the unique ability to make learning portable.25,33,38 However, it is the link with the final affordance, A5 (situated learning in context), which draws on the deeper meaning of AR where the experience is engaging meaningfully with the environment. However, this “environment” can be considered in a number of ways – as well as being a real rather than simulated environment4,25,35 it can also include real objects (such as physical bodies used in MM)26,28,29 or phantoms and mannequins.23,31,39,41

Conclusion

The systematic review described in this article reveals some significant findings in the area of AR in medical education. One of the reasons for undertaking this review was to ensure that the most recent work in this field was included, and it is notable that 11 of the 21 articles reviewed were from 2019 or 2020. This suggests that this area of work is highly current and of increasing interest to the medical profession. Some of the findings of this review reinforce those of previous reviews, for example the tendency to focus mostly on the areas of anatomy and surgery. However, it is notable that there are several other specialties covered by the selected articles and it may be that a rich area of future research is to invest more in new approaches to medical training using AR in areas beyond anatomy and surgery. We also note the broad range of AR tools that appear in these studies, ranging from low-cost tablet-based AR visualization through to complex multicomponent AR simulators, with a range of technologies integrated, including Smart Glasses, headsets and various projection technologies. In terms of our analysis of affordances, we note that studies that did not address any of the key affordances identified from the literature as being particularly relevant for the medical education context showed relatively poor learning outcomes. In contrast, studies that did leverage one or more of these affordances, particularly A5, which supports contextual learning, revealed positive outcomes. It is therefore suggested that paying close attention to the affordances of AR when designing systems for medical education may provide benefits.

Limitations and Future Work

There are some limitations to this study. The date range of selected articles was intended to ensure a contemporary view of the field but may as a result be excluding longitudinal trends. The criteria for selecting relevant populations may also be seen as a restriction on the results, in that otherwise relevant studies that related to experimental populations of “residents”, such as Mendes et al’s study on AR supported needle insertion44 were excluded. The length of medical training means that the term “medical student” is open to some interpretation, and in this study did not include residents. We note that in some studies, discussion of population was somewhat vague (eg, “novice”). We also note at the use of the term “augmented” is also used loosely in many studies. These issues of terminology can lead to inaccurate sampling of the literature. A further limitation of this study of that although it follows the structure of PRISMA it does not provide a separate qualitative synthesis and quantitative meta-analysis, focusing instead on the relationships between AR affordances and learning outcomes in different contexts.

Given the speed of technological change, future work might usefully repeat a similar study to include publications after 2020. A more traditional PRISMA qualitative synthesis and quantitative meta-analysis might also provide different perspectives.