Mixed Reality in the Reconstruction of Orbital Floor: An Experimental and Clinical Evaluative Study.

Introduction: Orbital floor fractures are common within midface fractures. Their management includes restoration of orbital volume and anatomy. However, these procedures could be associated with the mispositioning of implants and inadequate volume restoration. Nowadays medical rapid prototyping, virtual planning (VP), and navigation systems significantly increase the precision of such procedures. Nevertheless, the application of intraoperative navigation could be associated with intraoperative mistakes related to two-dimensional imaging. The application of mixed reality (MR) could solve this problem. The current study aims to demonstrate the application of MR in orbital reconstruction. Materials and
Methods: The current study included experimental and clinical implementation of MR in orbital reconstruction. Within the experimental part, 10 residents and 5 experienced maxillofacial surgeons were added. All data and customised software were well documented and then used in a single clinical case of orbital floor reconstruction.
Results: Visual assessment of plate positioning within the experiment revealed proper plate positioning in 8 cases. A comparison of virtual and real measurements showed a stable deviation of 0.65-1.15 (mean 0.9 mm). As a result of the clinical implementation of MR technology, after surgical reconstruction, the patient showed improvement in ocular mobility and reduction of diplopia. A postoperative computed tomography scan showed proper plate positioning. Discussion: Implementation of MR based on VP could significantly improve the results of preoperative planning, intraoperative navigation, and surgery. However, existing technical limitations that relate to navigation principles could produce mistakes and errors. Therefore, further investigations related to the 6 degrees of freedom problem solution are considered reasonable in the elimination of listed issues. Copyright:

INTRODUCTION

Orbits are one of the most common sites of midface fractures which account for approximately 40%.[1] These fractures are often associated with changes in orbital anatomy[23] that leads to changes in its pattern and volume. As a result, the orbital dimensions could be compromised by shifting the position of intraorbital contents, resulting in diplopia, enophthalmos, vision, and aesthetic disturbances.[24]

Nowadays for the improvement of functional and aesthetic outcomes, medical rapid prototyping (MRP) and intraoperative navigation systems are being introduced to orbital reconstruction.[238] Particularly, the application of MRP technology allows understanding of patient’s anatomy, perform orbital implants, and preplanning surgical procedures.[27] An intraoperative navigation system can use the radiologic and preoperative preplanning data as a map, to help position the reconstruction materials accurately, and decrease the rate of complications.[48]

Recently a new trend of mixed reality (MR) technology was introduced in different fields of surgery as pilot research. As a surgical navigation tool MR-based technology has been used for orthognathic surgery, face contouring, bone tumour resection, and neurosurgery.[131415161718]

The current study aims to demonstrate the possibilities of the application of MR technologies in orbital floor reconstruction.

MATERIALS AND METHODS

The current study was performed at the Department of Oral and Maxillofacial Surgery during the period of January 2017 to December 2019. The study design included both experimental and clinical implementation of holographic technology for the improvement of orbital floor reconstruction (all procedures performed in the study were conducted under the ethical standards given in the 1964 Declaration of Helsinki, revised in 2013, as well as the Ethical Committee of Azerbaijan Medical University, Faculty of Dentistry Protocol N 15).

A total number of 10 participants who are residents in the same department and 5 experienced maxillofacial surgeons were added to the experimental part of the study. All data and customised software were well documented and then used in a single clinical case of orbital floor reconstruction. The patient consent for the application of MR technology and publication was obtained before the procedure.

Experiment design

To achieve a “proof of concept” state of the study one proposed preclinical experiment. The random computed tomography (CT) scan data delivered from the database was imported to Mimics Medical 17.0 (Materialise, Belgium) software for further virtual simulation. Inclusion criteria were CT scans of the patients who were scanned for different reasons, while exclusion criteria included any pathology of the facial skeleton. The general aim was to simulate orbital floor reconstruction. Therefore, an artificial defect in the right orbit was created. Within the simulation, a virtual protocol that included the generation of a virtual orbital plate was conducted [Figure 1a-f]. All virtual procedures were done within 3-Matic Medical 9.0 (Materialise, Belgium) software.

Figure 1

Virtual simulation of orbital reconstruction: (a) importing of patient’s CT scan data; (b) cropping and simplifying of the virtual model; (c) creation of artificial defect in the orbital floor; (d) mirroring; (e) forming perimeter lines; (f) generation of the virtual orbital implant; (g) positioning of the virtual orbital implant-coronary view; (h) sagittal view; (i) 3D positioning; (j) generation of the scene; (k) reference points plate’s positioning assessment. CT: Computed tomography, 3D: Three-dimensional

Then virtual implant was loaded to Mimics Medical 17.0 software and appropriate positioning of the implant was achieved [Figure 1g and h]. The position and relationship to adjacent anatomical structures were double-checked in two-dimensional (2D) and three-dimensional (3D) views [Figure 1i].

As a next step, the obtained 3D data was used both for plastic model fabrication and virtual scene generation. The plastic model fabrication was done by Ultimaker Cura 4.1 software and Prusa i3 3D printer (Layer Height 0.06 mm, Infill 50%). The obtained plastic model was covered by a standard rubber mask. In addition, the double-sided tape was inserted within the region of the left orbital floor of the model to simulate plate fixation.

Generation of the virtual scene was done in Unity3d 2018.3.0 (Unity Technologies) by importing all components achieved from CT and virtual simulation [Figure 1j and k]. The main purpose of the developed software is to ensure the accuracy of the relationship between anatomical and virtual objects during the simulation. Custom C# scripts were used to control the simulation process. The user interface of the program includes two main components: one for turning on/off the virtual elements and the second to navigate virtual objects in 3D space and place them in the correct spot.

To achieve 3D mapping of the virtual object to a plastic model one used the principle of triangulation, which is the process of determining the location of a point by measuring only angles to it from the known points at either end of a fixed baseline, rather than measuring distances to the point directly as in trilateration. For these reasons, three cylinders were added both to the virtual and plastic model. The general idea was the superimposition of these cylinders in 3D space, and the achievement of accurate mapping.

Then all the data was loaded to Microsoft HoloLens MR headset (Microsoft Company, USA) to achieve 3D holograms in real space. The translocation and rotation of the holograms were realised by the usage of a clipper (a component of the HoloLens device) that works as a remote control.

Experiment

The experiment’s scenario included the placement of a preformed orbital mesh implant in the region of a defected orbital floor similar to a real-time procedure. Ten participants were asked to use Hartman alligator ear forceps and Farabeuf retractor to place the orbital implant into the orbit through a preincised rubber mask. The visualisation of the orbital floor was similar to what one can achieve within real-time surgery. For fixation of orbital plate, one used double-sided tape that was placed in the region of the defect of the orbital floor (being buried into the defect and therefore without production of any additional eminences). The plate was simply stuck to tape; therefore, no additional fixation was needed. The evaluation criteria were the position of the rear plate as compared to preoperative planning data [Figure 2a-d]. The evaluation was performed by experienced maxillofacial surgeons, who were provided with a preoperative CT scan and postoperative plastic model with the orbital implant. In addition, virtual and real measurements of plate position were done by means of a virtual and real caliper. As reference points one used distance from the infraorbital rim, lacrimal crest, and the apex of the orbit, relative to the plate’s terminal corners [Figure 1l]. Statistical analysis was performed by Microsoft Excel 2013 and MedCalc 12.7 software (P < 0.03).

Figure 2

Mixed reality aided surgical simulation: (a) mapping of the virtual hologram to plastic model; (b) insertion of the implant (side view); (c) insertion of the implant (see-through display view); (d) inserted implant

RESULTS

Experiment’s results

Subjective assessment of plate positioning revealed that the plate was positioned properly in 8 cases. The reason for inaccuracy in the rest of the 2 cases was the inability of participants to concentrate their attention on both holographic and real objects and the fact, that participants were inexperienced in gaming technology and augmented reality.

Objective assessment by the means of comparison of virtual and real measurements data showed inaccuracy in a range of 0.3–0.5 mm [Table 1 and Graph 1].

Table 1

Summary statistics table

	n	Mean	95% CI	Variance	SD	RSD	SEM	Median	95% CI	Minimum	Maximum	Normal distribution
A-A’	10	0.575	0.141-1.009	0.3677	0.6064	1.0545	0.1917	0.390	0.224-0.825	0.0500	2.100	0.0008
I-I’	10	0.356	0.0951-0.617	0.1330	0.3647	1.0244	0.1153	0.200	0.105-0.750	0.0400	1.060	0.1443
L-L’	10	0.334	0.00639-0.662	0.2097	0.4580	1.3712	0.1448	0.135	0.0747-0.648	0.0200	1.370	0.0099

SD: Standard deviation, CI: Confidence interval, RSD: Relative SD, SEM: Standard error of the mean

Graph 1

Hologram of statistical analysis

Clinical case presentation

The 38-year-old male presented with the symptoms of slight enophthalmos and significant immobility of the left eyeball in inferior and medial quadrants [Figure 3]. Subjectively, diplopia in all directions and paraesthesia on the upper lip left side were noted. Upon anamnesis, he had blunt trauma to the left orbit a month ago.

Figure 3

Clinical evaluation of eyeballs mobility: significant immobility was found in inferior and internal quadrants

CT scan revealed L-side “blowout” fracture [Figure 4a and b]. The inferior rectus muscle and medial wall of the left orbit were intact. 3D evaluation and orbital volume determination revealed its increase of up to 4.7 cc [Figure 4c]. Surgical reconstruction of the left orbital floor was chosen as a treatment option.

Figure 4

Virtual planning workflow: (a and b) blowout fracture of the left orbit; (c) explanation of orbital volume; (d) tracing of perimeter and guiding lines; (e) virtual plate generation; (f) virtual measurements. (g) Positioning of the virtual orbital implant-coronary view; (h) sagittal view; (i) 3D positioning; (j and k) generation of the scene. 3D: Three-dimensional

According to the clinical protocol accepted in our department, the virtual planning was performed and the patient-specific orbital implant was fabricated and its position to adjacent anatomical structures was checked in 2D and 3D views [Figure 4d-i].

Thereafter patient-specific scene was generated within customised software like the one used for an experiment that was described earlier [Figure 4j and k].

Surgical procedure has been done under GA through the transconjunctival approach. After exposure of the infraorbital margin, an accurate dissection within the orbital floor was carried out. Prolapsed soft tissues were extracted and returned to the orbit. Thus, the orbital floor and defected region were completely visualised. Once it was done, HoloLens holographic headset was placed on the surgeon’s head and the virtual scene was activated. After the mapping procedure, which was realised with the help of a clipper device (which was wrapped in a sterile sheath) the virtual hologram was superimposed on the patient’s head. As a reference, anatomical landmarks of both virtual and real teeth of the patient, as well as infraorbital margin were used [Figure 5].

Figure 5

The intraoperative wearing of the headset and see-through display view

Once the mapping procedure was completed the surgeon had the opportunity to verify the real orbital implant position through technology. In this step, the main task was the superimposition of the movable real patient-specific implant with a stable holographic implant. In this step, the surgeon was provided with the opportunity to change the intensity of hologram visualisation until the best fitting of real and virtual implants was achieved. Once superimposition was completed final fixation of the implant was done. On this step, the holographic smart glasses were taken off and a naked eye assessment of plate position as well as forced duction test was performed.

The postoperative course was smooth without complications. Soon after the surgical procedure, the patient showed diplopia related to the intra-periorbital oedema and temporary paresis of oculomotor muscles. A month later, the patient showed improvement in ocular mobility and reduction of diplopia. A postoperative CT scan a month after the procedure showed the plate to be placed properly in 2D and 3D sections [Figure 6]. However residual oedema was found in the region of medial and inferior rectal muscles. Therefore, the patient was referred to the Department of Ophthalmology for postoperative conservative treatment and observation.

Figure 6

Postoperative CT scan of the patient. CT: Computed tomography

DISCUSSION

Orbital “blowout” fractures are one of the most frequent types of fractures among all midfacial injuries.[19] Typically, clinical symptoms include evident enophthalmos, dystopia, diplopia of more than 2 weeks duration, and a forced duction test. Without adequate treatment, these fractures are showing a tendency for malunion and further changes in the position and integrity of the soft tissues.[20] Orbital reconstruction of its normal anatomy and volume in these cases is mandatory for the further prevention of listed symptoms.[2] However, postoperative complications are not rare and include residual diplopia and enophthalmos, infraorbital nerve dysfunction, which could be associated with misposition of the reconstruction materials.[3]

Nowadays progress of digital technologies allows for a significant increase in the accuracy of orbital reconstruction by application of virtual preplanning and MRP.[2345678] Moreover, the application of navigation systems leads to improvement of intraoperative positioning of implant materials, thus improving accuracy and diminishing mean surgical time.[48]

However, the application of this technology could be related to several technical problems, such as the necessity to switch the surgeon’s attention between the display of the navigating device and the time for system installation and patient positioning.[9]

As opposed to the traditional “head’s up” approach recently visualisation of navigation map could be executed by see-through display technology.[9] The concept of a headset with a superimposed display was introduced by Ivan Sutherland to the military in 1965 and consisted of a head-worn display and an image generation subsystem. However, these devices were cumbersome, heavy, and expensive that limited their application for health-care needs.[1011] Recently introduced less costly headsets such as Microsoft HoloLens could be successfully implemented in real-time surgery.[12]

Several publications described the application of mixed or augmented reality technologies in the field of plastic and reconstructive surgery. One could classify the range of applications of these technologies in 3 main directions: planning, navigation, and training.

In the case of preoperative planning in most of the publications, the general idea was to use the haptic device, immersive workbench, 3D eyewear, or 3D displays to simulate real-time surgery, thus finding the best option for every clinical case. This approach was used for preplanning of orthognathic surgery,[21] facial contouring surgery,[22] mandibular reconstructive surgery,[2324] cleft lip repair surgery, and craniomaxillofacial complex fracture reduction.[2526] Described technology was also used in hip fracture reduction surgery,[27] femur fracture reduction surgery planning,[28] orthopaedic fracture reduction surgery,[29] orthopaedic burring simulation,[30] and spinal fixation simulation.[34] In all cases, preoperative planning was done in real-time mode by virtual interaction of the operator through a haptic device and 3D eyewear with computer software.

As opposed to the presented technologies in our study, the hardware was limited to single 3D eyewear and a plastic model of the patient. Surgical planning was done before simulation and included virtual implant generation, implant template, and plastic model fabrication. In our opinion, this approach could simplify virtual planning workflow and reduce expensive hardware, such as haptic devices.

In cases of surgical navigation in most publications, the general principle included the usage of 3D patient mesh models that were reconstructed from CT or magnetic resonance imaging data and integrated into different AR devices, such as head-mounted displays, interactive portable displays with a camera, or tablet PC with an embedded camera. This technology was implemented for orthognathic surgery,[1314] mandibular angle osteotomy,[15] and tracking of maxillofacial bone, nerves, and vessels by application of markerless AR-based technology.[16] This technology was also implemented in bone tumour resection in the pelvic region.[17] However, authors reported difficulty in switching AR and VR display images that provide the closest distance information from a real surgical tool and a 3D patient mesh model.[1831]

The authors report that on average, participants made errors of up to 5.9 mm in length versus 2.8 mm during the naked eye tasks.[31]

In our study, navigation was provided by the application of the triangulation method that most probably leads to less error, with a maximum magnitude of up to 1.75 mm. Moreover, intraoperative navigation was achieved by multiple points (teeth and infraorbital margin) superimposition that makes the 3D hologram more stable. In addition, the accuracy of surgical procedures could also be increased by the tactile sensitivity of an experienced surgeon which was proved in our case.

Finally, in cases of surgical training, authors used both AR and VR technologies that included the application of the different haptic devices, immersive workbench, 3D eyewear, or 3D displays. These approaches were used for training in orthognathic surgery.[3233] AR/VR surgical training was also used in different surgical specialties such as training in orthopaedic surgery[34, 35] and in the field of neurosurgery.[36]

In our study of 3D eyewear and plastic, the model is similar to what was implemented for surgical simulation and planning. The application of a haptic device was not necessary, due to the experiment scenario that makes such technology less costly and therefore more applicable.

Another challenging problem that is related to the implementation of MR technology in medicine is the realisation of 6 degrees of freedom (6-DOF) which has received increased attention in the past few years.[3738] The main aim of the 6-DOF principle is to provide object tracking of typically small objects, such as patient anatomy or surgical instrumentation.[39] This kind of approach of combining navigation principles with MR will significantly increase the accuracy of surgical procedures and reduce operating time.[40] However, these approaches require the existence of surgical navigation systems, which makes this technology expensive.

CONCLUSIONS

Implementation of MR technology in combination with CAD could significantly improve the results of preoperative planning, intraoperative navigation, and surgical training. However, existing technical limitations within described methods are still producing several mistakes and errors in the application of such approaches intraoperatively. Therefore, improvement of MR technology that should be focused on the 6-DOF problem solution could be considered a reasonable way of eliminating of listed limitations.

Declaration of patient consent

The authors certify that they have obtained all appropriate patient consent forms. In the form, the patient(s) has/have given his/her/their consent for his/her/their images and other clinical information to be reported in the journal. The patients understand that their names and initials will not be published and due efforts will be made to conceal their identity, but anonymity cannot be guaranteed.

Financial support and sponsorship

This work was financially supported by the Ministry of Transport, Communications, and High Technologies of the Azerbaijan Republic.

Conflicts of interest

There are no conflicts of interest.