Application of Augmented Reality to Stereotactic Biopsy.

Both frame-based stereotaxy and frameless stereotaxy are established surgical procedures. However, they each have their respective disadvantages when used in the biopsy of a deep-seated lesion. To overcome the drawbacks associated with these procedures, we evaluated the feasibility of applying augmented reality (AR) to stereotactic biopsy. We applied our trans-visible navigator (TVN) to frame-based stereotactic biopsy in five cases of deep-seated lesions. This navigation system uses the AR concept, allowing surgeons to view three-dimensional virtual models of anatomical structures superimposed over the surgical field on a tablet personal computer. Using TVN, we could easily confirm a clear trajectory avoiding the important structures as well as the target point's location in the lesion. Use of the stereotactic apparatus allowed the surgeon to easily advance the biopsy probe to the target point. Consequently, a satisfactory histopathological diagnosis without complication was achieved in all cases. In conclusion, applying AR to stereotactic biopsy is feasible and may improve the safety of the procedure.

Introduction

Although frame-based stereotaxy is an established procedure used in the biopsy of deep-seated lesions,[1–3)] errors due to inaccurate registration, incorrect targeting, incorrect coordinates, and the patient’s head movement while in the frame may decrease the accuracy of the procedure. Once the apparatus is set, there is no way to reconfirm the accuracy of the target. Frameless navigation using an optical or electromagnetic localization system is an alternative method for stereotactic biopsy,[1–3)] but it requires an additional apparatus to advance the probe precisely to the target.[2–5)] Moreover, it requires surgeons to alternate their viewpoint from the operative field to the monitor. Augmented reality (AR) allows a real-time updated 3D virtual model of anatomical structures to be superimposed over the surgical field.[6)] It has been successfully used in neurosurgical procedures including aneurysmal clipping,[7,8)] brain tumor resection,[7,9–11)] external drainage,[12)] pedicle screw placement,[13)] and percutaneous vertebroplasty.[14)] We applied AR navigation to overcome the disadvantages of stereotactic biopsy.

Surgical Techniques

The application of AR navigation to stereotactic biopsy was performed as part of a project to develop a tablet personal computer (PC)-based AR navigation system.[9)] This project was approved by the Jichi Medical University Clinical Research Ethics Committee. All patients gave written informed consent for their participation in the research.

AR-based navigation

Trans-visible navigator (TVN) is AR-based navigation using a tablet PC (Surface Pro; Microsoft, Redmond, WA, USA; Fig. 1A). The development of our TVN system was previously described in detail.[11)] Briefly, TVN allowed the superimposition of 3D virtual models over the surgical field and captured them using a back-facing camera on a tablet PC (Figs. 1B and 1C).

Fig. 1

Trans-visible navigator (TVN) consists of augmented reality (AR) navigation using a tablet personal computer (PC) (A). TVN shows three-dimensional virtual models superimposed over the actual surgical field captured with a back-facing camera on a tablet PC (B and C).

Augmented reality application was programmed using Unity Pro (Unity Technologies, San Francisco, CA, USA) and then installed into the tablet PC. Graphic resolution of the TVN system was 1920 × 1080 pixels.

A 3D position measurement was obtained using the motion capture system (VICON, Oxford, UK), and head registration was achieved using the anatomical landmark method. The 3D virtual models were created on the basis of preoperative computed tomography (CT) and magnetic resonance images using an imaging software (Amira; FEI, Hillsboro, OR, USA). We created 3D virtual models of the skin, bone, tumor, vein, and venous sinus in all cases and the brain ventricles in one case.

Surgical procedure

Target coordinates were determined on contrast-enhanced CT images scanned in the stereotactic frame (Komai Stereotactic Frame; Mizuho Ltd., Tokyo, Japan). In the operating room, the patient’s head and stereotactic frame were fixed with a Mayfield Ultra Base Unit. The patient’s head was registered to the TVN using three anatomical landmarks: the nasion and bilateral preauricular points. Next, the surgeon determined the puncture trajectory and burr-hole position with the aid of TVN, which superimposed 3D virtual models over the surgical field to avoid important anatomical structures within the trajectory. After the burr hole was drilled, the stereotactic frame was attached and set with predetermined coordinates (Fig. 2A). Before puncturing, the surgeon reconfirmed the location of the trajectory within the lesion and the absence of structures in the trajectory path by manually aligning the camera axis of the tablet with the puncturing trajectory of the stereotactic frame. The biopsy probe was then advanced to the target point using the stereotactic apparatus (Fig. 2B).

Fig. 2

The combination of trans-visible navigator (TVN) and stereotactic frame realized augmented reality-guided biopsy. Target coordinates were determined based on contrast-enhanced computed tomography images scanned in the stereotactic frame (A). The puncture trajectory was determined using TVN (B).

Case Presentations

In five cases this surgical procedure was used (Table 1). In all cases, the CT images taken immediately after the procedure confirmed a trajectory and the biopsy point located in the lesion. Histopathological diagnosis of the five cases included glioblastoma in two, malignant lymphoma in two, and anaplastic astrocytoma in one.

Table 1

Series of trans-visible navigator guided stereotactic biopsy

Patient no.	Age (years)	Tumor location	Diagnosis	Complication	Registered models	Imaging modalities used to create the 3D models
1	79	Right frontal lobe	Glioblastoma	None	Tumor, vein, venous sinus	MRI, CT
2	78	Left temporal lobe	Glioblastoma	None	Tumor, vein, venous sinus	MRI, CT
3	60	Right occipital lobe	Anaplastic astrocytoma	None	Tumor, vein, venous sinus	MRI, CT
4	52	Right cerebellum	Malignant lymphoma	None	Tumor, vein, venous sinus	MRI, CT
5	76	Right thalamus	Malignant lymphoma	None	Tumor, vein, venous sinus, ventricle	MRI, CT

CT: computed tomography, MRI: magnetic resonance imaging, 3D: three-dimensional.

Illustrative case 1, patient no. 5

This patient had a right thalamic tumor (Fig. 3A). This case required a puncture, avoiding the ventricle and bridging vein. In addition, the hair line was located near the surgical field. Consequently, a burr hole was drilled behind the hair line for cosmetic reasons.

Fig. 3

A right thalamic tumor (A). In this case, the trajectory required avoiding the bridging vein and ventricle in addition to selecting a puncture site inside the hair line. Trans-visible navigator allowed the surgeon to confirm the trajectory by superimposing three-dimensional virtual models over the surgical field (B). Before puncturing, the surgeon reconfirmed the puncture trajectory using the probe eye view (C).

Trans-visible navigator showed a 3D virtual model of the bridging vein, ventricle, and tumor superimposed over the surgical field (Fig. 3B). Therefore, we were able to select the optimal puncture trajectory to avoid important structures with consideration of the hairline (Fig. 3C).

Illustrative case 2, patient no. 4

The patient had a right cerebellar tumor. Because the tumor was situated in the posterior fossa, special care must be taken due to the position of the sinuses. Therefore, the sinus, emissary vein, and the tumor were registered using TVN and viewed on the tablet PC during surgery. Following AR visualization, we obtained a good orientation, even after covering the head (Fig. 4A). Therefore, a safe trajectory and burr-hole position were possible using TVN guidance (Fig. 4B).

Fig. 4

The case of right cerebellar tumor. Augmented reality visualization using a three-dimensional virtual model allowed good orientation, even after covering the head (A). The puncture trajectory was planned to avoid the sinus and emissary veins and was confirmed using the probe eye view (B).

Discussion

Our TVN allows surgeons to reconfirm the accuracy of the target point and suitability of the trajectory by superimposing 3D virtual models over the surgical field, avoiding the drawbacks of frame-based stereotactic navigation. Our TVN also allows surgeons to advance the biopsy probe precisely along the trajectory without turning their view-points from the surgical field to the monitor, avoiding the drawbacks of frameless stereotactic navigation. The tablet does not interfere a surgeon’s view because real-time surgical field can be displayed on the tablet via its camera. In addition, there is sufficient space for surgical manipulation. Our study demonstrates that applying AR to stereotactic biopsy is feasible and may improve the safety of the procedure.

The limitations of our TVN for stereotactic biopsy are threefold. First, it is sensitive to the issue of brain shift, although it is common for all navigation methods and not limited to TVN.[15)] Second, although it is an easy procedure to perform, it requires an additional 30 min to create the 3D models and registration. Third, TVN did not show the real-time position of the needle tip in our surgical system. Therefore, TVN guided frameless biopsy has still not been achieved.

In conclusion, we tested the feasibility of applying AR navigation to stereotactic biopsy. The use of this navigational approach may improve the safety of the procedure.