Collaboration of AR Device and Separable Two-layered Elastic Models as Tools for Surgical Education.

Three-dimensional printing can aid trainees in gaining a better appreciation of different anatomical structures and pathological conditions.[1] Furthermore, these three-dimensional models permit intensive training before an operation. We made realistic three-dimensional computer-assisted two-layered elastic models[2] and separable models[3] of the face. Residents and young doctors could experience realistic simulated surgery and understand three-dimensional movement of the flaps.

However, Oren et al reported on an augmented reality device for improving intraoperative judgment/workflow and also mentioned the possibility of surgical education and remote communication.[4] HoloLens from Microsoft (Microsoft Corp, Redmond, Wash.), which is a head-mounted mixed reality device, can display a precise three-dimensional model stably on the real visual field as a hologram. Using the HoloLens, we devised an effective application for simple manual alignment of surgical field and holograms.[5]

The three-dimensional printing models and the AR technology each have their own advantages and disadvantages. Therefore, we are considering using both 3D models and AR technology in a mutually supportive manner. The purpose of this study was to assess whether residents and young surgeons are able to learn more effectively by using both these tools in combination when they practice simulation surgery. First, a senior surgeon created a bilobed flap on the above-mentioned separable two-layer model, and then the surgical processes were captured by three methods: digital camera (2D image); VECTRA 3D camera (3D image); and video. A trainee surgeon then wore the HoloLens and performed the same surgical operation while projecting the example image in each of the three ways (Fig. 1). We examined the usefulness of this method as an educational tool and compared the different methods of projection. (See Video [online], which shows an actual field of view when wearing the HoloLens.)

Fig. 1.

A trainee surgeon designs a skin flap on a two-layered elastic facial model while projecting a hologram with HoloLens.

Video 1.

Field of view when wearing the HoloLens. Video 1 from “Collaboration of AR Device and Separable 2-layered Elastic Models as Tools for Surgical Education”

{L-End}

The 2D image projection was considered useful for copying the senior surgeon’s design due to the excellent visibility of the overall shape and markings (Table 1). On superimposing the 3D images, we used the above-mentioned application.[5] A feature of the 3D image projection method is that 3D positioning details are easily grasped and can be overlapped. This allows the comparison with the surgeon’s design and results by superimposition on the model. The video projection method allows the user to directly visualize a series of actions, making it possible to verify and study detailed movements that are difficult to discern from still images. This method is therefore recommended for use when confirming the details of surgical techniques.

Table 1.

Advantages and Disadvantages of the Three Projection Styles

	Advantages	Disadvantages
2D image	1. It is easy to grasp the whole object.2. The size of the hologram can be adjusted to that of the actual model.	1. The hologram cannot be projected at the same angle as the model on the desk.2. The hologram cannot be superimposed onto the model.
3D image	1. It is easy to grasp the details such as three-dimensional positioning.2. The hologram can be superimposed onto the model.	1. It is difficult to grasp the whole object due to the appearance of shadows.2. Techniques are required to superimpose the hologram onto the model.
Video	1. It can be used to check the sequence of actions.2. Visual observation of the motion is possible.	1. The gestured instructions for playing and stopping the video are difficult to convey.

While the 3D models have the advantage of being touchable and may be manipulated before the operation, the AR devices cannot be used to practice techniques by themselves. However, their greatest advantage is that they add additional information to the visual fields. Although the multiple-layered complicated models cannot be made because of their technology and cost, residents can obtain additional information by holograms on the two-layered models or the simple 3D bony solid models using the AR devices. For these reasons, we believe that this collaborative method of the AR device and 3D models can serve as a better educational tool and can compensate for the disadvantages of the other method.

DISCLOSURE

The authors have no financial interest to declare in relation to the content of this article.