Creation of a video library for education and virtual simulation of anatomical lung resection.

OBJECTIVES: Recently, preoperative and intraoperative simulation using three-dimensional computed tomography (CT) has attracted much attention in thoracic surgery. However, because conventional three-dimensional CT only shows static images, dynamic simulation is required for a more precise operation. We previously reported on a resection process map for pulmonary resection, which we developed to generate virtual dynamic images from preoperative patient-specific CT scans. The goal of this study was to evaluate the feasibility of the clinical use of the resection process map for anatomical lung resection.
METHODS: This study included 5 lobectomies for different lobes and 4 representative segmentectomies. Dissection of the pulmonary arteries, veins and bronchi were considered key parts of each procedure. To assess the description of images obtained from the resection process map, relevant clips from the actual surgical videos were collected, retrospectively replicated and superimposed on the resection process map to explain the procedures.
RESULTS: In all surgical procedures, the resection process map successfully and semiautomatically generated a virtual dynamic image from the patient-specific CT data. Moreover, superimposition of the virtual images on the selected clips from the surgical videos showed no major differences.
CONCLUSIONS: The resection process map could generate virtual images that corresponded to the actual surgical videos and has the potential for clinical use as preoperative and intraoperative simulation.

INTRODUCTION

Video-assisted thoracoscopic surgery (VATS) has been widely performed as a minimally invasive operation, especially for early-stage lung cancer [1-4]. Although VATS is relatively less invasive, it requires more complex technical skills and limits palpation during the operation, owing to the small incisions for the ports. Moreover, with this current VATS-oriented trend, junior surgeons may have few opportunities to perform open surgery and more difficulty in understanding detailed anatomy, given that VATS provides only a two-dimensional (2D) view of the surgical field through the scopes and monitors used during the procedure.

Simultaneous with the development of the VATS approach, three-dimensional (3D) computed tomography (CT) was frequently used for preoperative or intraoperative simulation and has helped surgeons understand the anatomical relationships and plan strategies [5-11]. Although 3D-CT is one of the most innovative technologies in the field of thoracic surgery in the last 2 decades, there is still room for improvement. Because the lung deforms dramatically during an operation, more precise dynamic simulation of this intraoperative deformation would be helpful, compared with conventional 3D-CT, which can show only static images. In addition, virtual reality simulators that were designed for technical training for VATS lobectomy have been developed [10-12]. Although these simulators were validated in some studies, only built-in cases are available. Because the anatomical variations differ among patients, a technology that allows more precise preoperative simulation using patient-specific radiologic data would be useful.

From this perspective, we developed the software resection process map (RPM), which can generate a virtual dynamic image based on patient-specific CT images, for simulation of pulmonary resection [13-15]. In a previous study, we were convinced that the novel simulation system could quickly generate accurate, deformable images for anatomical lung surgery [15]. Because this system might be expected to work as a tool for both preoperative and intraoperative simulation, in this study, our goal was to validate whether the virtual images could be merged into the actual surgical videos.

MATERIALS AND METHODS

Construction of virtual dynamic three-dimensional images

This study was approved by the institutional review board of Kyoto University (R1420).

At the Department of Thoracic Surgery of Kyoto University, a 3D-CT image analysis system (SYNAPSE VINCENT, Fuji Film Co, Ltd., Tokyo, Japan) was used to reconstruct the CT images of each patient and, subsequently, to perform lobar and sublobar segmentation. According to the lobe or segment designated by the surgeons for resection, the system extracted information on the pulmonary structures and displayed 3D static images. After extracting the surface data for the 3D images in the stereolithography file format, the 3D-CT image analysis system was operated using another single computer terminal. A computer with graphic processing units (Intel Corei7 3.60 GHz with 32.0 GB memory; Intel Corporation, San Francisco, CA, USA or NVIDIA Ge-Force GTX 1070; NVIDIA Corporation, Santa Clara, CA, USA) was used to convert the stereolithography data into the final virtual dynamic image form. In all these steps, the system generated a virtual image semiautomatically. Because the virtual image was dynamic, it was able to provide views of the 3D model from arbitrary directions, could be manipulated by a mouse, and could mimic some procedures, such as opening of the fissure and intersegmental plane, dissection of the hilum and arbitrary traction of the lung. After being deformed to replicate the surgical video, the virtual images were extracted [15].

Surgical procedures

We collected representative cases of 5 lobectomies, which included the right upper, middle and lower lobes and the left upper lobe, and 4 segmentectomies, which included the right S2 and S6 and the left S1 + 2 and S3. Cases performed with VATS were preferred over those performed with open surgery, because the former had video recordings of the techniques available throughout the procedure. Critical parts of each procedure, such as dissection of the pulmonary arteries, veins and bronchi, were selected from these videos.

Superimposition of the actual videos and virtual images

In the activity of superimposing the virtual image extracted from the RPM and the surgical video, we focused on using the virtual image to show the detailed anatomy. To better clarify and understand the intrathoracic orientation, we first adopted the virtual image showing the whole lung instead of the usual figure descriptions.

Next, superimposing the virtual image in the video was the key to description in this video. Vessels and bronchi were enhanced by the layered virtual image. Especially for the complex structure in the hilum, the 3D description of the virtual image was done by changing the colour tone of the vessels and bronchi; the colour tone was made deeper for the front structures, whereas shading was used to designate the structures on the back.

The workflow for creating the fused images is shown in Fig. 1. We organized the photographs of the surgical clips, the whole-lung virtual images that depicted the location of the procedure and other virtual images that mimicked the surgical clips. The representation of each structure and the permeability of the lung in the virtual images were adjusted to be descriptive.

Figure 1:

The workflow for creating explanatory images from conventional three-dimensional computed tomography.

RESULTS

For each procedure, we collected images of the key intraoperative graphics, virtual images that were replicated for the surgical clips, and a virtual image of the whole lung. In addition to the static images, an explanatory video was successfully created by superimposing the actual images and the virtual images (Figs  2 and 3, Supplementary Material, Figs S1–S6 and Video 1).

Figure 2:

Images from a right upper lobectomy. The column on the far left shows key clips of the dissection of the pulmonary vein (a1) and artery (b1) and the bronchus (c1). The cells in the next column show overlaid images of the surgical pictures and virtual images (a2, b2 and c2). The cells in the third column, a3, b3 and c3, represent virtual images of the relevant area of the surgical field. The column on the far right shows the whole lung according to each procedure, and the white box indicates the area where the procedure is done (a4, b4 and c4).

Figure 3:

Images from a left S1 + 2 segmentectomy. The row on the far left shows key clips of dissection of the pulmonary vein (a1) and artery (b1) and the bronchus (c1) from the top to the bottom. The cells in the next row show overlaid images of the surgical pictures and virtual images (a2, b2 and c2). The cells in the third column (a3, b3 and c3) represent virtual images of the relevant area of the surgical field. The column on the far right shows the whole lung according to each procedure, and the white box indicates the area where the procedure is done (a4, b4 and c4).

Figure 2 shows a representative case of a lobectomy in the right upper lobe and demonstrates dissection of the pulmonary vein and exposure of the frontal hilum by pulling back the lung (a1). The position of the lung and the angle of thoracoscopic view are shown in Fig. 2a4 by the virtual image of the whole lung; the scope of the targeted area is shown in the white box. The a3 cell is an enlarged view of the relevant area. In a2, the superimposition of the actual surgical image and the corresponding virtual image clearly demonstrate the lineage of the hilar vein. Similarly, the interlobar view of the pulmonary artery and bronchus was provided. Virtual images were prepared based on the surgical clips that showed adjustment of the thoracoscope angle and the fissure opening. Thereafter, the exposed anatomy of the A2, A6 and upper lobe bronchus was described by overlay (b3).

Figure 3 shows a representative case of segmentectomy in the left S1 + 2. As described in the lobectomy case in Fig. 2, the hilum view with the dissected pulmonary vein was selected first as one of the key clips for this operation. In Fig. 3, V1 + 2 and V3 are visualized (a2). Next, interlobar clips are described with an opened fissure; the interlobar pulmonary arteries (A1 + 2c, A4 + 5 and A6) and bronchi (B1 + 2) were selected, considering that these 3 clips were dynamic during the flow of the procedure when the lung was pulled in different directions. A virtual image was created to replicate the clips that showed deformation of the whole lung during the procedure and the corresponding surgical field. Superimposing the pictures from the surgical video on the virtual image led to the understandable depiction of the anatomical structures.

Video 1 shows a representative case of a 48-year-old woman who was referred to our institution because of a small nodule in the left S3. She underwent a VATS left S3 segmentectomy. The lesion was pathologically diagnosed as lung metastasis from breast cancer. In this video, the process of pulling back the lung to expose the surgical field was described first and was followed by the virtual video showing the hilum dissection. Next, dissection and division of the pulmonary vein and pulmonary artery were shown; virtual objects were overlaid to show the structures clearly. After dissection of the hilum, an interlobar view was provided to show a sequential surgical video and virtual images of opening the fissure. In this view, we explained that the pulmonary artery coursed through the middle of the interlobar space and described the 3D structure that comprised the pulmonary artery and bronchus. A virtual image was added to aid in the understanding of the detailed anatomy pertaining to the dissection of the B3 bronchus, which was not clearly delineated in the surgical video. After dissection of the bronchus, the view was placed back to the hilum, where the complex structure that comprised the pulmonary artery, vein and bronchus was exposed. Here the procedure in which B3 was encircled and divided is shown.

One point that should not be ignored was that the virtual dynamic images depicted inflated lungs because the usual CT images were used by the RPM. In addition, in some cases, peripheral vessels may be seen on the surgical video to extend out from the real lung parenchyma, because the virtual images were based on CT images of an inflated lung.

DISCUSSION

In a previous study, we employed finite element modelling, which can perform fast computations, to develop a new system that could enable surgeons to interactively simulate in real time anatomical lung resection manoeuvres such as holding, grasping and traction [15-17]. The RPM can be manipulated by a mouse and was equipped with unique functions, such as (i) provision of 3D views from arbitrary directions; (ii) lung traction, which is similar to hilum dissection or opening the fissure or the intersegmental plain; (iii) visualization of vascular structures and bronchi throughout the lung parenchyma; (iv) mimicking the deformation of vessels and bronchi along with the lung; and (v) replicating changes in the RPM in response to each manipulation without delay. We revealed that the RPM could accurately and rapidly generate virtual dynamic images from patient-specific preoperative CT data [15]. However, this system needed validation of concordance with the surgical videos, considering its possible clinical use in the future. In this study, we tried further retrospective implementation of the RPM by superimposing the videos and virtual images of the designated technique. In all 9 cases of anatomical lung resection, the virtual images could be merged into the actual surgical videos without distinct differences. This result indicated that the RPM has the potential to be used as part of preoperative simulation and intraoperative navigation.

Another strong point of this study was the educational aspect of superimposing the images, which provided explanatory descriptions of the anatomy and the crucial intraoperative technique. Because learning from 2D surgical videos of VATS may be difficult, the focus of this study was to show the depth of each structure in the virtual images without the need for any special equipment for 3D visualization. Although we acknowledge 3D VATS and robotic surgery are already available, in which surgeons are provided with a sense of depth by 3D vision, we have focused on the use of the RPM in 2D VATS first because conventional 2D VATS is considered to be used in most facilities. Enhancing the cognition of anatomy could complement the experience and shorten the learning curve of young surgeons. Moreover, inclusion of the main and important clips from each operation may enable trainees to effectively learn the procedures outside the surgical environment. For example, if we encounter a case of anomaly of vessels or bronchi, we can build a 3D image from the RPM and store it in a library for future reference. Furthermore, the concept of RPM and creating a library may be applicable to surgery on other organs, such as the liver and kidney. Moreover, our technology can be integrated into virtual reality simulators for VATS training and can provide more anatomical variations and even opportunities to simulate upcoming cases using virtual images generated from actual CT data.

In this study, we realized that surgical procedures were composed of several key clips. For example, a right upper lobectomy comprised the following 3 clips: anterior hilar, posterior hilar and interlobar. We have also provided several key clips from different surgical procedures in the figures and supplementary files. Based on these representative clips, we may be able to hasten the process of creating the important RPM image corresponding to the key surgical step; this procedure has the potential of being the first real-time navigation for clinical use in the field of thoracic surgery.

Limitations

This study has some limitations. First, the virtual images were generated from inflated lungs. We are now exploring and validating the prediction of lung deflation for RPM use [18]. Although some vessels were shown to extend beyond the lung parenchyma, it was not important to explain this because each main procedure was performed mainly in the hilar area of the lung. Second, the simulation applications were retrospective. To evaluate the efficacy of preoperative and intraoperative simulation, prospective analysis should be performed in a future study. Furthermore, once the effectiveness of the RPM as a simulation software is demonstrated in clinical use, cost-benefit issues might be discussed not only for certified thoracic surgeons but also for surgical residents.

CONCLUSION

The RPM could generate virtual images that corresponded to the actual surgical videos and has the potential for clinical use in preoperative and intraoperative simulations.

SUPPLEMENTARY MATERIAL

Supplementary material is available at ICVTS online.

Click here for additional data file.