Immersive photoreal new-age innovative gameful pedagogy for e-ophthalmology with 3D augmented reality.

Augmented reality (AR) has come a long way from a science-fiction concept to a science-based reality. AR is a view of the real, physical world in which the elements are enhanced by computer-generated inputs. AR is available on mobile handsets, which constitutes an essential e-learning platform. Today, AR is a real technology and not a science-fiction concept. The use of an e-ophthalmology platform with AR will pave the pathway for new-age gameful pedagogy. In this manuscript, we present a newly innovated AR program named "Eye MG AR" to simplify ophthalmic concept learning and to serve as a new-age immersive 3D pedagogical tool for gameful learning.

The Inception of the Pedagogical Application “Eye MG AR”

Throughout the last decade, augmented reality (AR) has slowly and steadily become a substantial part of modern life, with increasing applications in the field of medicine and ophthalmology.[1] During COVID-19, the impetus of AR into ophthalmology is more stronger than ever. The COVID-19 pandemic has momentously changed the landscape in which we practice ophthalmic medicine. As we adapt to what is the new “norm” surrounding social distancing and interpersonal contact, innovative ways to teach and learn e-ophthalmology are in high demand.[2] Hence, we have innovated an AR program named “Eye MG AR” to simplify ophthalmic concept learning and to serve as a new-age immersive 3D pedagogical tool for gameful learning.

Gamification - 3D Gameful Learning Tool

Recently, an AR program named “Eye MG AR” was innovated by us to show different three-dimensional (3D) anatomical structures [Figs. 1 and 2] related to e-ophthalmic learning, with multiple customized angles of the viewer’s choice to simplify concept learning, using real-time TrueColor confocal images. Simple structures such as the eyeball [Fig. 3] and complex structures relevant to the eye (cerebral venous and dural sinuses) were also constructed in advanced real-time 3D photoreal visuals for immersive visual experiences.[34] This pedagogical transformation in e-ophthalmology aims to reinvent the approach to ophthalmic teaching through virtual platforms with Unreal Engine software [Figs. 4-6] [Video Clip 1 and 2]. This AR app was constructed with pioneering ways of cult teaching on an android platform, along with gameful 3D learning in ophthalmology. Gamification in e-learning with deep visualization and cinematic experiences has never been reported in the literature and can pave the way for a new age ophthalmic pedagogy.

Figure 1

Image showing annotated three-dimensional (3D) model of the eyeball

Figure 2

Image showing the annotated 3D side view of the cerebral venous and dural sinuses model

Figure 3

Image showing the pedagogue viewing the eyeball model: (a) top view; (b) anterior segment; (c) posterior segment

Figure 4

(a) Logo of the AR software (Unreal Engine). (b) Main-level workspace in the Unreal Engine software. (c) Cerebral venous and dural sinuses model in the viewport created by us. (d) Level blueprint created by us in the software. (e) Widget blueprint created by us in the software

Figure 6

Image showing the cerebral venous and dural sinuses model used inside unreal engine workspace

Image showing the eyeball model used inside Unreal Engine workspace

Setting Up the Framework for Accessing Eye MG AR

With the following steps [Video Clips 3 and 4] one can access this gamification tool for infinite e-learning:

Download the AR template and take a print of it and place it on your table.

Install the Eye MG AR app on your android phone from the Google Play Store.

Launch the app and focus your mobile camera [Fig. 7] on the AR template [Fig. 8], which should be fitted within the given borders shown on the display screen [Fig. 9].

Figure 7

Image showing the pedagogue beginning the augmented reality (AR) teaching session by focussing the mobile camera over the AR template

Figure 8

Image showing the AR template to be downloaded and printed as the base, for the model to appear

Figure 9

Image showing the borders within which the AR template should be fitted in the mobile screen

Anatomical structure of the eye/cerebral and dural venous sinuses [Figs. 10 and 11] appears.

Figure 10

The pedagogue viewing the eyeball (red arrow) in the mobile screen over the AR template

Figure 11

The pedagogue viewing the cavernous sinus (red arrow) in the mobile screen over the AR template

Rotate the paper to access customized degrees or angles for the user’s view of the various structures.

Move the mobile accordingly to view the structure from the top [Fig. 12].

Figure 12

Top view of the cerebral venous and dural sinuses when mobile is adjusted accordingly

Pedagogical Transformation

To make ophthalmic concept learning better, we have innovated this app for studying ocular anatomy, built on an advanced interactive 3D touch interface. Concepts pertaining to ophthalmology that have lots of theoretical framework can be broken into small fragments, as no longer the neophytes have to mentally visualize them during training [Fig. 13] [Video Clip 5]. This is a powerful cognitive tool that serves as a 3D encyclopedia, where students can choose their optimal frame of choice, from the 360° angle for viewing [Video Clip 6-9], thus filling in their cognitive mental gaps.

Figure 13

Image showing the pedagogue. (a) Training the students using the Eye MG AR app. (b) Demonstrating the parts of the eyeball. (c) Focusing the optic disc

Prerequisites of the App

This app is currently available free of cost from the Google Play Store for Android phones that support Google Play services for AR.[5] The users can either download the app from the Google Play Store or directly scan the QR code [Fig. 14] and then download it. More ocular structures related to ophthalmology are being added as updates for comprehensive ophthalmic pedagogy. To support this AR app, the android device must have a compass (magnetometer), a rear-facing camera, a gyroscope, a global positioning system (GPS), and an accelerometer.[6] These devices need to run Android 7.0 Nougat or later, and need to be installed with Google Play services for AR. In this era, there is no dearth of smartphone users. The majority of students have already been using their smartphones to take lecture notes, and for surfing the internet for studying. Carving a mobile application with 3D AR will allow them to not have any dark side regarding the anatomy of oculus uterque. The app is currently under development for iOS users.

Figure 14

Image showing details of the Eye MG AR app on the Google Play Store, and the QR code for directly downloading the app by scanning it

Conclusion

AR enhances the real world with immersive experiences by providing a digital overlay over the real world. In contrast, virtual reality replaces the real world and shuts out the physical world. Ophthalmology is a field of medicine that is well suited for the application of AR. We believe that time will indeed foster further technological advances and lead to widespread use of AR in routine ophthalmic practice.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.