Augmented reality and dynamic infrared thermography for perforator mapping in the anterolateral thigh.

Dynamic infrared thermography (DIRT) has been used for the preoperative mapping of cutaneous perforators. This technique has shown a positive correlation with intraoperative findings. Our aim was to evaluate the accuracy of perforator mapping with DIRT and augmented reality using a portable projector. For this purpose, three volunteers had both of their anterolateral thighs assessed for the presence and location of cutaneous perforators using DIRT. The obtained image of these "hotspots" was projected back onto the thigh and the presence of Doppler signals within a 10-cm diameter from the midpoint between the lateral patella and the anterior superior iliac spine was assessed using a handheld Doppler device. Hotspots were identified in all six anterolateral thighs and were successfully projected onto the skin. The median number of perforators identified within the area of interest was 5 (range, 3-8) and the median time needed to identify them was 3.5 minutes (range, 3.3-4.0 minutes). Every hotspot was correlated to a Doppler sound signal. In conclusion, augmented reality can be a reliable method for transferring the location of perforators identified by DIRT onto the thigh, facilitating its assessment and yielding a reliable map of potential perforators for flap raising.

INTRODUCTION

Several studies have investigated the preoperative identification of perforators. A wide range of methods have been reported, including computed tomographic angiography (CTA), magnetic resonance angiography, acoustic Doppler ultrasonography, color duplex ultrasonography, fluorescent angiography, and near-infrared spectroscopy [1]. Recently, thermography has gained increased attention [2-5], mainly because of its ease of use and the possibility of adopting smartphone-based cameras to reduce the cost of implementation [6]. Dynamic infrared thermography has also been described for the preoperative mapping of perforators, and it has been suggested to be correlated with intraoperative findings [5]. This method relies on assessing the skin after a cooling and reheating cycle [7-9]. A cooling process followed by a recovery protocol before acquiring the thermal image can greatly enhance the visualization of the perforators. Larger-caliber perforators appear first after reheating the skin and are represented by larger hot spots [5]. Additionally, some authors have suggested the possibility of identifying linking vessels between the perforators by analyzing the reheating pattern [10].

However, for an accurate reading, certain environmental conditions, such as the room temperature, should be standardized. Moreover, the precooling process can be difficult to implement in the operating room, and relying on preoperative markings can cause details about the reheating pattern to be missed. Additionally, in our experience, it can be inconvenient to watch the thermal camera screen while trying to match the hotspots on the skin surface. To address these problems, we developed a simple method of using augmented reality in which the thermal image is acquired in optimal environmental conditions, using a smartphone-based thermal camera, and is then projected back onto the skin using a portable projector, which can be done in the operating theater.

The aim of this study was to evaluate the accuracy of the thermographic projections of perforators identified using an acoustic Doppler device.

IDEA

This study was reviewed and approved by the Institutional Review Board (IRB) of Pontificia Universidad Católica de Chile (IRB No. 170331005). The IRB waived the requirement for written consent.

The setup consisted of a thermal camera (Therm-App, Opgal Optronic Industries, Karmiel, Israel) attached to a smartphone (Samsung Galaxy S3; Samsung Electronics Co. Ltd., Suwon, Korea) through a mini USB (universal serial bus) connector (Fig. 1). The camera’s standard software was replaced with an improved commercial version (ThermAppPlus) in order to adjust the color palette range.

Fig. 1.

Thermal camera setup

(A) Anterior and (B) posterior view of the thermal camera attached to a smartphone.

To test the accuracy of our method, we assessed the anterolateral aspect of six thighs in three healthy volunteers. Verbal informed consent was obtained from the participants. The subjects were allowed to rest comfortably without any clothes covering the thigh for 5 minutes at room temperature (20°C–22°C). We manually focused the 19-mm lens of the thermal camera on the volunteer’s thigh, at a distance of 1 m. With aluminum foil, we marked the anterosuperior iliac spine, the lateral patella, and the midpoint between them, after which we cooled the thigh skin for 2 minutes using a desktop fan blowing air at room temperature (Fig. 2). Generally, the skin temperature would drop by 1°C to 2°C. The software’s color palette was set with its higher and lower values at 1°C over and under the skin temperature at the end of the cooling process, respectively. After 3 minutes of rewarming at room temperature, a real-time video was captured, from which a representative picture was taken using the embedded software and stored on an secure digital (SD) card. The picture was later transferred to the portable projector. The perforators could be identified as hotspots, while the aluminum foil was seen as a cold spot in the thermal images (Fig. 3). We assessed a 10-cm diameter area around the midpoint as our region of interest. After acquiring the picture, we projected the image back onto the thigh, taking care to align the cold spots with the areas covered by aluminum foil (Fig. 4). The hotspots projected onto the skin were assessed using an acoustic Doppler device (Supplemental Video S1). We identified a median of 5 perforators (range, 3–8) in the proposed area, and the median time needed to identify the perforators was 3.5 minutes (range, 3.3–4.0 minutes). All perforators in the six thighs projected onto the skin had an audible Doppler signal, with a 100% correlation.

Fig. 2.

Anterolateral aspect of the thigh

Aluminum foil marks (asterisks) are present in the anterosuperior iliac spine, the lateral patella, and the midpoint between them.

Fig. 3.

Thermal image of the anterolateral thigh

Red spots (white arrows) represent the hotspots. Black spots (white asterisks) represent the aluminum foil, due to its lower temperature.

Fig. 4.

Thermal image projected over the thigh surface

DISCUSSION

The use of thermography for preoperative perforator mapping has received increased attention. We agree with other authors that pre-assessment cooling and reheating can optimize the visualization of perforators.

Using portable cameras [11] and smartphone-based cameras [6] is advantageous due to their portability. However, caution should be taken regarding the accuracy and sensitivity of the camera [12]. We selected the Therm-App (Opgal Optronic Industries) camera over other cheaper cameras because of its superior sensitivity (<0.07°C) and resolution (384×288 pixels).

The idea of using augmented reality for reconstructive surgical planning has been previously explored. A near-infrared vein visualization device has been utilized for the selection of a recipient vein with an intact valve in lymphatic supermicrosurgery [13], and abdominal CTA-reconstructed images have been projected over a patient’s abdomen to provide a visual aid while planning a deep inferior epigastric artery perforator flap [14].

We present a simple and consistent method that uses augmented reality and thermography to transfer the perforator mapping to the skin. This method can be used in the operating theater before flap harvesting, with no need for a cooling challenge or optimizing the room temperature.

At the beginning of this study, we projected a real-time video over the thigh surface using an intranet connection to another smartphone that was actually connected to the projector. This method yielded a dynamic real-time image of the vascular network. In the clinical setting, this method is highly useful because it allows us to capture the hotspots in ideal thermal conditions (not present in the operating theater) and to project them into the operative field. This can provide an accurate mapping of the vascular network of each patient. Virtual reality and augmented reality are emerging technologies that have found their way into plastic surgery, mainly in the contexts of preoperative planning, navigation, and simulation [15]. We hypothesize that having a visual guide to multiple perforators projected over the body’s surface could have a real impact on the surgeon’s decisions regarding flap design. For instance, the surgeon might select a larger hotspot or design the flap over a higher density of hotspots, ultimately leading to fewer perfusion-related problems and lower costs.

In all our measurements, the hotspots projected over the skin correlated with Doppler sound signals. Consequently, a very small amount of time was needed to confirm their presence. It should be noted that this method relies on adequately identifying anatomical landmarks and matching them with the thermal image.

Additionally, the possibility of viewing the thermal cooling and reheating in real time could be helpful for analyzing the possible presence of linking vessels between perforators, as proposed by Chubb et al. [10].

Nonetheless, we encountered some limitations in our study. In order to adequately see the overlapping image on the skin, the room had to be dim, due to the low brightness of the portable projector. Additionally, we had to use a camera stand in order to steadily align the thermal cold spots with the anatomical landmarks. Finally, the same author acquired the image and assessed the Doppler signals. Thus, our study was not blinded in terms of the evaluator.

Some authors have proposed that larger and brighter hotspots correlate with larger vessels, due to the warmth of blood flow. Additionally, stronger Doppler signals have also been suggested to be correlated with larger perforator vessels. Because of the camera resolution, in some occasions we could observe hotspot clusters, which were most likely terminal branches of a single perforator reaching the dermis. These areas could easily be mistaken for a large hotspot. When assessed with the Doppler device, they showed heterogeneous signal strength along the hotspot.

Care should be taken when using a previously recorded image, due to the dynamic nature of skin flow. However, the location of perforators should remain the same.

In this study, we present a technique for acquiring thermal images and projecting them back onto the skin surface as a technique for preoperative perforator mapping. This method uses an affordable thermal camera and a portable projector, which is likely to be readily accessible. To the best of our knowledge, this is the first report of combining dynamic infrared thermography and augmented reality for the identification of perforators. Future studies should use thermography to assess the functional aspects of the perforators, such as perfusion area and flow.