Smartphone-assisted Augmented Reality in Craniofacial Surgery.

Augmented reality (AR), a blending of both the physical and digital worlds, can be a valid tool for surgeons wishing to plan interventions and attain symmetry. The use of technology has enabled physicians to achieve desirable results. In this article, we describe a method that uses smartphone's simple AR utilities for convenient, cost effective, and time saving perioperative planning.
METHODS: Images of preoperative computed tomography, along with 3D reconstructed scans were uploaded to a smartphone and used in an affordable application (Camera Lucida) to superimpose the loaded pictures over the smartphone camera. In one case, a 3D computed tomography scan of the skull was mirrored to help guide fronto-orbital advancement, and in another case the loaded 3D reconstructed computed tomography scan was used to prioritize areas of scalp coverage in a complicated case of craniosynostosis with major scalp wound dehiscence. DISCUSSION: Adaptation of AR to assist in the field of craniofacial surgery has been introduced before in several studies that reported the use of computer-based guidance for cranial reshaping. The majority of these reports used sophisticated modalities, combining advanced image registration and tracking with specialized equipment. The utility of smartphone AR for cranial vault reconstruction provided good accuracy when visualizing fronto-orbital advancement and remodeling, together with in depth prioritization of areas in need of soft tissue reconstruction.
CONCLUSION: Smartphone AR adaptation proved to be a very convenient tool assisting in the planning of different craniofacial conditions that are time saving and do not incur any additional fees beyond those of the surgery.

INTRODUCTION

Craniosynostosis is defined as premature fusion of the cranial suture, and has an incidence of one in every 2500 births.[1,2] Different types of craniosynostosis are defined based on the fused suture; including coronal, sagittal, lambdoid, metopic, and sphenofrontal. Patients presenting with this pathology can be experiencing one type of suture or fusion of multiple sutures. Premature fusion of these sutures can result in abnormal growth of the skull.[3] Surgical intervention by means of cranial vault reshaping with possible fronto-orbital advancement depends on the fused suture’s location and is typically performed in the first year of life.[2,4] Craniosynostosis can result in distinct clinical features that might represented a sincere concern for patient’s family members, in addition to having a psychosocial impact on the developing child.[5]

Technology’s integration with medicine has increased and been refined over time. This integration has contributed to different aspects of surgical fields, and is continuing to aid physicians in accomplishing desirable outcomes. Augmented reality (AR), which blends the physical and digital worlds, offers surgeons a unique option for planning various reconstructive techniques. The term AR implies superimposition of a digital image on a user’s view of the world, and by that providing a composite view.[6] AR was first utilized in the medical field in the early 1990s, with its application in the contexts of preoperative surgical planning and analysis of complex data arising intraoperatively.[7] Symmetry is a desirable feature linked to a good cosmetic appearance, and as a result improves the psychosocial status of a patient.[5] The utility of different AR modalities was tried in different craniofacial interventions in which it has assisted in the perioperative planning of cranial vault reshaping, where it helped to ensure a higher level of symmetry.[5]

In this article, we report our experiences with the use of an affordable means of AR using a smartphone, to show how these modalities have assisted in the planning of cranial vault reshaping and fronto-orbital advancement perioperatively, together with their potential application in the management of complications associated with different pediatric craniofacial conditions.

METHODOLOGY

Images of preoperative computed tomography (CT), along with three-dimensional reconstructed scans, were uploaded to a smartphone, and then opened in smartphone image superimposition software. We have used an affordable application (Camera Lucida), in which the application superimposes the uploaded picture on top of the smartphone’s camera view. The application then allows the user to adjust the transparency of the image, as well as the tilt, saturation and position of the projected image. Other features of such applications include mirroring and positioning, which were also beneficial for this project. The uploaded CT images serve as a guide that can be aligned to the actual patient camera view. The superimposed images were aligned to the patient utilizing the antroposterior skull length (frontal to occipital) and bitemporal skull width as anatomical reference points to guide for proper alignment. We used this technique in the surgical planning of two pediatric craniofacial cases. The first case was an isolated sphenofrontal craniosynostosis, which presented with ipsilateral forehead flattening and dystopia. The second case was that of a patient who had undergone total calvaria reconstruction in a different institute, complicated by wound dehiscence with exposed critical structures.

Case 1

A 14-month-old boy was referred as a case of an isolated right sphenofrontal craniosynostosis involving flattening of the ipsilateral forehead and a shallow supraorbital margin. The patient showed no signs of increased intracranial pressure or papilledema. The patient’s family members were concerned about the asymmetry and requested surgical intervention to correct this deformity.

Surgical Procedure

The patient was brought to the operating room and placed in a supine position. Under general anesthesia, a standard zig-zag bi-coronal incision was marked and the flaps elevated until we accessed the supraorbital rim. We then utilized the smartphone application that had been loaded with the patient’s preoperative 3D reconstructed CT scan viewed in a position similar to the patient’s position, and then we aligned to the patient anatomical reference points. The CT image helped demonstrate the pathology of the ipsilateral flattening of the forehead and the supraorbital margin. We designed the osteotomy markings, and a cut was made in the frontal bone, followed by an osteotomy for the right unilateral fronto-orbital bar, all performed in the usual fashion. With the assistance of the smartphone application, the normal side was mirrored on the diseased side and then the result superimposed on the patient’s camera visualization of the actual deformity (Fig. 1). We then analyzed and estimated the advancement required to attain the maximal symmetry possible to match the normal side, through a trial involving matching the contralateral supraorbital rim and the forehead. The advanced right unilateral fronto-orbital bar and the reconstructed frontal bone were then stabilized with absorbable plates and screws. Finally, we used the AR application to compare and confirm the advancement carried out before skin closure, which showed elimination of the forehead discrepancy, and that the advanced unilateral fronto-orbital bar matched the contralateral side with good symmetry. Subsequent follow-up examination showed good symmetry, and the family reported high satisfaction.

Fig. 1.

The utility of smartphone AR in fronto-orbital reshaping. A, Preoperative 3D reconstructed CT image with isolated sphenofrontal deformity. This image was utilized in the AR process where it was first mirrored horizontally to itself then superimposed on top of the smartphone camera view, as seen in B. The difference in the fronto-orbital bar is clearly seen, differentiating the patient’s current state and the mirrored normal side. Visualization of the reconstructed fronto-orbital bar in relation to the normal side, as seen in C and D. The final intraoperative result obtained before skin closure is shown in E. Preoperative and 1-year postoperative follow-up patient photographs are shown in F and G.

Case 2

A 23-month-old girl with pan-synostosis of cranial sutures and with cognitional ichthyosis, who had undergone total cranial vault expansion for high ICP, and was referred to our center after developing postoperative skin dehiscence at the surgical site, followed by necrosis at the scalp flap edges. The patient was referred for management of dehisced and necrosed areas that showed exposure of bone and critical structures underneath. A preoperative CT scan was performed to identify the exposed areas and to delineate the cranial reshaping completed elsewhere, after which the decision was made to take the patient for definitive coverage.

The patient was brought to the operating room and placed in a supine position and under sterile technique. The smartphone application was loaded with the patient’s CT 3D reconstructed image, which was superimposed on the patient’s head to identify areas that lacked bone flaps and to determine where brain was exposed. The flaps were designed in consideration of the 3D image superimposed on the patient’s skull, giving priority to ensuring skin flap coverage for areas that were not covered with bone (Fig. 2). (See Video [online], which shows the process of superimposing and fitting the loaded CT scan image of patient’s different anatomic parts over the smartphone camera view, with the feasibility of adjusting the transparency of the projected image to facilitate image alignment.) Three transpositional flaps were designed based on named vessels using a hand-held Doppler for confirmation. The flaps were transposed in a manner to ensure proper coverage of exposed areas of brain. Raw areas of exposed bone with intact periosteum after flap mobilization were covered with a split thickness skin graft. Postoperatively, the patient was found to be doing well, with healed wounds, and no soft tissue complications.

Fig. 2.

The utility of smartphone-assisted AR in planning soft tissue coverage of a complicated craniosynostosis case with multiple wound dehiscence. A, Superimposed 3D reconstructed CT images of the patient’s skull with the previous cranial reshaping. B, Planned three transposition flaps that were designed prioritizing the areas denuded of bone. C, Flaps raised as planned.

Video 1.

The process of Superimposing and Fitting the Loaded CT Scan Image over the Smartphone Camera View. The video shows the process of fitting and aligning the viewed image over the patient’s different anatomic parts, with the feasibility of adjusting the transparency of the projected image to facilitate image alignment.

DISCUSSION

With the recent advancements in technology, AR methods have been implemented in a variety of health care specialties. Such technologies provide a means by which immersive imagery can serve as an informative guide for each patient interactively.[8,9] The adaptation of this technology to the field of craniofacial surgery has been investigated previously, and several studies have reported the use of computer-based guidance when performing cranial reshaping. However, the majority of these reports used sophisticated modalities with advanced image registration and tracking, together with specialized equipment.[10-12]

Craniosynostosis patients typically present with a deformity that results in discrepancies in the skull shape. This skull distortion is a genuine concern for the patients’ family and has potential psychosocial effects on the patient and their family.[5] Cranial vault reconstruction helps by dramatically correcting the deformity; however, the amount of symmetry varies despite this being the surgeon’s chief aim.

Numerous articles have described the use of augmented and virtual reality when performing calvaria reshaping. For example, in craniomaxillofacial surgeries, 3D analysis can significantly enhance different reconstructive approaches through intraoperative imaging of a construct based on preoperative 3D models.[13,14] Moreover, virtually planned osteotomies for fronto-orbital remodeling and advancement have produced excellent results and are also characterized by ease of use.[15] Furthermore, 3D AR images can enhance surgery results in craniofacial surgery by assisting surgeons when forecasting desirable results.[16]

Perhaps one of the most challenging factors in the utility of smartphone assisted AR use for perioperative cranial vault planning is image alignment and tracking. To address this, different authors have proposed the use of occlusal splints and printed guides.[17] Additionally, the utility of smartphone AR by means of image superimposition over the camera view have been investigated before for its use in cranial vault reconstruction. The technique utilized has shown good accuracy when visualizing osteotomy guidance together with fronto-orbital advancement. However, this adaptation required the use of customized 3D printed guides together with 3D photography to accurately register the tracking point for proper AR alignment,[17] all of which increase the cost of the procedure and the time spent on planning.

The utility of simple smartphone AR was investigated in a way that involves employing various reconstructive specialties. This involves simple image superimposition of the patient’s radiological workup, for example in the case of perforator-based flaps and lymphatic reconstruction. This method was validated and found to have comparatively good accuracy levels relative to other planning modalities.[18,19]

We adopted the same technique but for a different surgical approach in which we utilized an affordable smartphone-based AR in the reconstructive planning for different craniofacial conditions. Perhaps one of the confounding variables was the complex alignment axis that was fixed. This was achieved by taking photographs and positioning the 3D reconstructed CT image in the same position as the patient on the operative table, after which we aligned the superimposed image on the patient in camera view to different anatomic reference points (the antroposterior skull length “fronto-occipital and bitemporal skull width as anatomical reference points). This was associated with good image alignment. We have used the smartphone capabilities in one case in which the contralateral normal side was mirrored horizontally and used as a reference point for assessing the symmetry of the correction (case 1) and/or the underlying bone defect to anticipate areas that would need coverage (case 2) as seen in the Supplemental Video. (See Video [online].) The method proved to be a simple and convenient tool to assist with the planning and placement of reconstructed structures.

The use of the smartphone AR application adds minimally to the expense of the surgery. In addition, it is a simple tool to use and does not require the assistance of a technician or sophisticated equipment. However, the application does not precisely inform or give data regarding the osteotomies’ location and length, but the surgeon can comprehend and estimate the symmetry attained by comparing with the superimposed CT image when it is viewed on the patient. For that, and compared with other modalities, computer assisted virtual reality provides precise data about osteotomy locations and height, but adds notably to the surgical cost.[15,16]

The aim of the study reported here was to shed light on an alternative and convenient utilization of AR when planning different reconstructive craniofacial surgeries. AR applications available via smartphones provide a variety of options for reconstructive intervention, including planning the advancement needed for a fronto-orbital bar to correct a supra-orbital rim asymmetry, frontal bone reconstruction to eliminate forehead discrepancy if normal reference point is there, or in the arrangement of scalp transpositional flaps to ensure the coverage of desired areas as presented in this report. This, however, is limited in the information it provides, like the osteotomy placement, the amount of advancement needed or when the visualization of the deformity in more than one access in which interactive alignment and image superimposition is lost.

CONCLUSIONS

The use of smartphone-aided AR proved to be a very convenient method for planning various craniofacial reconstructive interventions. The technique is economical in terms of both time and financial cost. Furthermore, it allows surgeons to anticipate desirable results, although it is not entirely free of limitations.