An augmented reality system in lymphatico-venous anastomosis surgery.

Indocyanine green lymphography, displayed as infrared image, is very useful in identifying lymphatic vessels during surgeries. Surgeons refer the infrared image on the displays as they proceed the operation. Those displays are usually placed on the walls or besides the operation tables. The surgeons cannot watch the infrared image and the operation field simultaneously. They have to move their heads and visual lines. An augmented reality system was developed for simultaneous referring of the infrared image, overlaid on real operation field view. A surgeon wore a see-through eye-glasses type display during lymphatico-venous anastomosis surgery. Infrared image was transferred wirelessly to the display. The surgeon was able to recognize fluorescently shining lymphatic vessels projected on the glasses and dissect them out. Published by Oxford University Press and JSCR Publishing Ltd. All rights reserved.

Introduction

Identifying lymphatic vessels is a key step in lymphatico-venous anastomosis. Because lymphatic vessels are thin and fluid inside is transparent with faint color, it is not easy to spot them with bare eyes. Indocyanine green (ICG) lymphography with infrared camera is one of the most powerful tools for identifying lymphatic vessels. Most of those camera systems have visual displays, which are usually placed on the wall or besides the operation tables. During the identification procedure, the surgeons have to move their viewpoints back and forth from the surgical field to the display. This sequence disturbs the surgeon’s concentration.

An augmented reality (AR) system with a see-through glasses type display was developed to see the infrared view and identify lymphatic vessels.

Case Report

A 74-year-old lady who had underwent hysterectomy, has been suffering for bilateral lower extremities lymphedema for 30 years. She wished to undergo lymphatico-venous anastomosis. On the day before the surgery, 0.2 ml of 0.25% ICG was injected subcutaneously into the first web spaces, bilaterally. The operation was performed under general anesthesia. An infrared camera (W134B, Watec, Tokyo, Japan) was placed upside down, on the other side of the patient from the surgeon. The camera was connected to a personal computer for recording and displaying. The operator wore an eye-glasses type display, Moverio BT-200/AV (Epson, Suwa, Japan). Infrared image was wirelessly transferred through high-definition multimedia interface port and wireless monitor adaptor (Fig. 1). Fluorescently dyed lymphatic ducts were identified and dissected out by the surgeon, watching both infrared and bright field views through the glasses (Fig. 2). Lymphatico-venous anastomosis was performed under a microscope (Fig. 3). The patient continued wearing compression stocking after the surgery. Perioperative images and circumferential lengths of the patient are shown in Fig. 4 and Table 1.

Figure 1:

Schematic drawing of the AR system. An infrared light and camera were placed upside down, on the other side of the patient from the operator. A laptop computer was to record and display. Though a mirroring adaptor, the image was wirelessly transmitted and displayed on a see-through glass type display. The operator saw the infrared image overlaid on the real scene.

Figure 2:

A surgeon is dissecting out lymphatic vessels, referring infrared image projected on a see-through glasses type display.

Figure 3:

A lymphatic vessel (right side) anastomosed to a venule (left side).

Figure 4:

(a) A 74-year-old lady with bilateral lower extremities lymphedema before lymphatico-venous anastomosis. (b) Six months after the surgery.

Table 1:

Perioperative circumferential lengths of the patient’s lower extremities.

(cm)	Circumference of	Upper thigh	Lower thigh	Ankle	Foot
Pre-operation
	Right	44.0	40.5	24.5	22.0
	Left	47.0	37.5	23.0	21.0
6-months post-operation
	Right	44.0	33.0	24.0	20.0
	Left	46.0	30.5	21.5	19.5

Discussion

To deal with lymphedema on extremities, several remedies have been employed. In addition to conservative treatment, such as compression stocking and massage, surgical treatment has been evolved. Lymphatico-venous shunting was developed to drain lymphatic fluid. At first, it was done macroscopically. Stamp of half cut lymph nodes was sutured on venous walls [1]. Patent blue dye was used to spot lymphatic vessels. Lymphatic vessels were identified as appendant to the lymph nodes. Then, microscopic lymphatico-venous anastomosis emerged [2].

ICG is a green dye that has long been used as test reagent for liver function. Motomura et al. [3] used ICG to identify sentinel lymph nodes during breast cancer surgery, watching with bare eyes. Though, green color of ICG is very faint in lymphatic vessels. ICG binding to some kinds of proteins, absorbs infrared rays and emits fluorescence. Ophthalmic angiography with ICG [4] has been routinely done with this characteristic features. Nimura et al. [5] used infrared ray electronic endoscopy to detect ICG dyed sentinel lymph nodes in gastric cancer surgery. Ogata et al. [6] introduced this ICG dying and infrared viewing into lymphatico-venous anastomosis surgery. Infrared viewing is also useful in examining blood vessels [7].

To see infrared view, cameras and displays are indispensable. The displays are usually placed besides the operation table or on the wall of the operation room. To identify lymphatic vessels in the infrared view and dissect out them in the operation field, the operator has to move his or her viewpoint from the display to the operation field, back and forth. This necessitated cycle annoys the surgeons.

The term ‘AR’ is attributed to Thomas P. Caudell [8]. It is a technology that allows computer-generated information overlaid onto to the real world in real time [9].

With our AR system, infrared view is overlaid on real see-through view. So, the surgeon can see both views simultaneously, without moving his or her head in macroscopic dissection. Once the lymphatic vessels are identified, microscopic lymphatico-venous anastomosis can be done.

At this moment, there still are several points to be improved. The glass display unit weighs as light as 88 g, though the surgeon wearing it felt it heavy. There is a time lag between real movement and the displayed image. The infrared image captured with the camera was processed with a laptop computer and wirelessly transferred to the glasses type display. Speeding up throughput speed may shorten the time lag, though there may be some limit. View point of the camera is away from the surgeon’s eyes. So, infrared view cannot be perfectly overlaid with the real image. Position alignment also cannot be perfect. Utilizing camera on the display unit may solve the problems. Development of AR technology can be of great help in surgery fields.