Mechanism and design of a novel 8K ultra-high-definition video microscope for microsurgery.

The practical application of microscopes using 8K ultra-high definition (UHD) technology is progressing. However, due to insufficiencies in factors such as luminous intensity and stereopsis, it has not been possible to achieve sufficient image quality for close observation of submillimeter order microlymphatic anastomosis using a combination of 8K-UHD cameras with a rigid endoscope. We have improved the quality of microsurgery by the introduction of a new heads up 8K-UHD surgical system. Herein, we show the mechanisms of this next-generation technology that makes optical improvements to the electronic image input data, resolving the initial drawback. We have developed a new 8K-UHD digital microscope system with digital zooming to enable maximum 300X magnification of the surgical field. This system has specific lighting settings for shadows dropped in surgical field to expand the three-dimensional effect while still being a monocular camera. The original mechanism and design enable the increase of the depth of field with optimal angles between the imaging direction and approaching direction towards the surgical field. Assessment during a pre-clinical trial using rats demonstrated that it is possible to perform microlymphatic anastomosis in a heads-up position with a 70-inch 8K-UHD monitor and the 8K-UHD monocular camera system. Performing supermicrosurgery is difficult with conventional surgical microscopes. Our results illustrate the application of this new 8K-UHD microscope system to this new field.

Introduction

In conventional microscopic surgery, the resolution of surgical views through eyepieces is limited by both the performance of the optical lenses and visual acuity of the surgeon. Optical microscopes are designed for observation through eyepieces by the surgeon's naked eyes. Therefore, the specifications of optical lenses are designed to increase human eyesight. The quality of the surgical view obtained from a surgical microscope is limited by the surgeons' visual acuity. In order to solve conventional surgical microscopic issues such as limitation of surgeons' visual acuity and surgeons' unnatural posture for extended periods of time, methods for heads-up surgery with a video microscopic system have been introduced [1].

Almost all recent video microscopic systems for heads-up surgery have, at most, 4K ultra-high definition (UHD, 3840x2160) resolution [2, 3], but not 8K UHD (7680 × 4320 pixels) resolution.

Heads-up surgery provides the advantage of sharing the microsurgery view with not only the surgeon and assistant surgeon, but all medical staff [1]. For observation of the highest resolution microsurgery view, the largest possible display should be near the surgeon, as the high resolution of a small display cannot be fully appreciated in the operating room due to the limitation in the visual acuity of surgeons. By contrast, a large display should be kept away from the surgeon to distance the display from the operating table. A large display needs to have the highest possible resolution to provide an advantage for heads-up microsurgery. In order to convert conventional microsurgery to heads-up microsurgery, a higher resolution is required. 8K-UHD can provide the highest quality image with 60 fps, the same as the conventional surgical video rate.

In a previous study, we developed a lightweight 8K-UHD camera and conducted a feasibility study to test the utility of the 8K-UHD microscope in an ophthalmic surgery on pig cadaver eyes [4]. The 8K-UHD microscope comprised of a surgical microscope (Carl Zeiss, Zeiss Lumera T, Oberkochen, Germany), a camera adaptor with relay lenses, an 8K-UHD camera, and an 8K-UHD liquid crystal display (LCD) (LV-70002; Sharp Corporation, Osaka, Japan), which shared the 8K-UHD images in real time. However, super-enlarged images produced by a monocular 8K-UHD camera and optical lenses in the microscope have narrow depths of field and a low three-dimensional effect. This initial trial of 8K-UHD microscope could not be applied in supermicrosurgery. Thus, we have improved the quality of microsurgery by the introduction of a new heads up 8K-UHD surgical system [5]. Herein, we show the mechanisms of this next-generation technology that makes optical improvements to the electronic image input data, resolving the initial drawback.

Materials and methods

The new 8K-UHD camera combined with an optical lens

The basic premise of the 8K-UHD camera with a rigid endoscope for microscopic use is illustrated in Figure 1A and the new 8K-UHD camera with a macro lens in Figure 1B. Previous 8K-UHD images were captured using our former endoscopes design [6, 7, 8]. In these cases, a relay lens connects the target side to the 8K-UHD camera. Although the amount of light in the field of view cannot be increased with the previous design, the newly developed 8K-UHD camera with a macro lens is designed to increase the diameter of the optical lens to allow the introduction of sufficient light. Sufficient light leads higher image quality. In addition, larger lens provides less distorted images, and therefore a large macro lens is needed for 8K-UHD microsurgical imaging.

Figure 1

Differences between our two 8K-UHD cameras and lens optical systems. (A) Our conventional 8K-UHD camera with a small endoscopic relay lens for microscopic use. (B) Our new 8K-UHD digital microscope with a large macro lens. For 8K-UHD imaging, large optical lenses are desirable with sufficient light producing less distorted images. UHD, ultra-high definition; ISO, International Organization for Standardization.

Specifications of the camera and lens of our new 8K-UHD digital microscope system are shown in Figure 2 and Table 1. Figure 2 shows the relationship between the observation distance and resolution of the 8K-UHD microscopic camera and lens. In this graph, the size of the field of view is fixed to the entire ISO resolution chart. In order to generate enough working space without changing the target field of view, our system enables optimal combination of exchangeable single focus lens and digital zooming ratio. However, longer focal lens length and higher zooming ratio produces lower image resolutions. In addition, lowering the lens aperture from F5.6 to F16 caused a decrease of image resolution, where F is the lens focal length divided by the diameter of the entrance pupil of the lens. Nevertheless, our system was able to keep working distance constant while retaining over 4K-UHD resolution using all lenses and digital zooming with the exception of F16.

Figure 2

Relationship between observation distance and resolution of the 8K-UHD microscopic camera and exchangeable lens with digital zooming ratios. The field of view size is fixed to the entire ISO resolution chart regardless of observation distance, lens in use, and digital zooming ratio. Our system can keep working distances constant while retaining over 4K-UHD resolution using any lens and digital zooming ratio except for F16. UHD, ultra-high definition; ISO, International Organization for Standardization.

Table 1

Specifications of the 8K-UHD camera.

Item	Specification
Camera resolution	7680 (H) x 4320 (V) pixels
Image sensor	Super 35 mm mono CMOS, 3.2 μm × 3.2 μm
Digital zooming ratio	1.0–4.0 times
Focal length of lenses	100 mm, 140 mm, 200 mm
F number of lenses	5.6~
Minimum focus distance	30 mm

CMOS, Complementary Metal Oxide Semiconductor.

Features of the 8K-UHD video microscope

The new microscope is capable of the following three points.

A maximum of 300X magnification with digital zooming

The magnification ratio of a conventional surgical microscope is approximately 20–40X. We obtained a maximum of 300X magnification for microsurgical images by using a 4.0X digital zoom of the camera when using a 200 mm macro lens from a distance of 30 cm. Standard surgical microscope has optical zooming lens. However, optical zooming lens for 8K-UHD imaging is not practical because lens size becomes extremely large in contrast with the camera head size. Therefore, we adopted digital zooming image processing with high resolution single focus lens between x1.0 and x4.0. Maximum digital zooming provided magnification display of a 2K resolution area on the whole 8K-UHD monitor. Using the exchangeable optical lenses and digital zooming function, the 8K-UHD image, which is approximately 300 times the actual field of view size, can be displayed on a 70-inch 8K-UHD monitor (LV-70002; Sharp Corporation, Osaka, Japan). Therefore, we observed not only the capillaries on the lymphatic vessel but also the blood flow through the capillaries.

Improvement of the three-dimensional effect on monocular images

In short-distance observation, it is reported that even monocular observation can provide three-dimensional effect with information such as “occlusion”, “binocular disparities”, “motion perspectives”, “relative size”, “convergence and accommodation” and “relative density” [9]. In surgical microscopy, surgeons can catch depth perception by placing a needle and thread before the target organ or tissue or overlapping the left and right instruments. In addition, “shading” and “shadow” provided by illumination are effective for a sense of unevenness and position in the field of surgical view.

In conventional surgical microscopes, the imaging and illumination direction are uniform to remove unnecessary shadow in the surgical field. However, when securing the three-dimensional effect of objects, it is unfavorable to completely remove the shadow of objects. Without the shadow of objects in the surgical view, it is difficult to recognize the positional relation of the shapes of lymphatic vessels, needles, and threads. Our system can emphasize the monocular three-dimensional effect of shadows from objects in the surgery field using an inclination of illumination direction from the imaging direction (Figure 3). The size and overlapping of objects, as well as the shadow dropped from objects and overlapping of the shadows, help with recognizing positional relations between objects in the surgical field with a monocular image.

Figure 3

Schematic of the dropped shadow effect created by inclination of illumination direction from viewing direction with a monocular 8K-UHD camera. The shadows make it easy to perceive positional relations between objects in the surgical field. UHD, ultra-high definition.

Increase in depth of field

In conventional microscopic use, imaging direction and approach direction of surgical instruments are uniform. In order to increase depth of field, we adopted the inclination of the 8K UHD camera imaging direction from the approaching direction (Figure 4). The angle between the imaging direction and the approaching direction towards the surgical field with surgical instruments such as forceps and needle holders is θ. This angle θ makes the approaching distance towards the surgical field cosθ times in the imaging direction. The depth of field in a surgical operation is increased by 1/cosθ times to prevent objects such as lymphatic vessels, needles and threads in the surgical field from being out of focus (Figure 4). Moreover, adjustment of the diaphragm of the lens can increase the depth of field.

Figure 4

The depth of field can be increased by changing the inclination of the 8K-UHD camera imaging direction from the approach direction of surgical instruments. The angle θ between both directions makes the approach distance towards the surgical field cosθ times in the imaging direction. The original depth of field L is increased by 1/cosθ times to prevent surgical instruments in the surgical field from being out of focus. UHD, ultra-high definition.

In vivo test

We performed a preliminary clinical test with the above optical settings (Figure 5). The study was conducted using four 200–250 g male Lewis rats. Under inhalation anesthetic, a ventral incision was made and tweezers were used to carefully expose the abdominal lymphatic vessels at the inferior extremity of the left kidney. The in vivo test of the system was approved by the Keio University School of Medicine Animal Ethics Committee (Approval No. 18011). This study was conducted in compliance with all the applicable institutional and the national ethical guidelines for the care, welfare and use of animals.

Figure 5

The 8K micro video camera can be set up at angles of approximately 30° in relation to the surgical field. The surgeon performs the surgery in a seated position with a frontal view in the “heads-up” position on a large screen.

Results

Increase in depth of field by angling the imaging direction

We used a lens focus calibration tool alignment ruler folding card (DSLRKIT Photography and Network Equipment) to measure the depth of field of the 8K-UHD microscopic image. Figure 6 shows the relationship between the F-number of the lens, angle between the imaging direction and approaching direction, and depth of field. Furthermore, adjustment of the diaphragm from F8 to F16 provided over two times the depth of field. Larger angling provided more than twice the depth of field. However, the gap became larger between the approaching direction and the imaging direction, causing lower operability. A larger F-number provided a larger depth of field. However, the resolution of the image became lower. This relationship is a trade-off, and therefore our system needs a suitable combination of angling of the camera and F-number of the lens.

Figure 6

Relationship between F-number of lens, angle between imaging direction and approaching direction, and depth of field. The depth of field increases by angling the imaging direction and a larger lens F-number.

Magnification of images in in vivo animal experiments

We observed microscopic surgical images with an adjustable magnification ratio of 300X with a digital zooming function of 4.0X by using optical settings such that the working distance between the lens and surgical field is 300 mm, a lens focal length of 200 mm, and an 8K-UHD LCD size of 70 inches. We were also able to shift between an ultra-magnified view of lymphatic vessels and a wide-angle view of the whole surgical field, including surgical instruments (Figure 7). This allows for obtaining larger magnification images with shorter working distances, which is a significant advantage of using an 8K-UHD video microscope.

Figure 7

Magnification of microscopic surgical view of a rat lymphatic vessel using only digitally zoomed 8K-UHD images. (A) is a default 1.0X wide-angle view of surgical field. (B) is a 2.0X magnified view. (C) is a 4.0X magnified view. We can easily observe a 0.3 mm lymphatic vessel close-up for microscopic surgical operation. UHD, ultra-high definition.

Discussion

Our results show a detailed report of the mechanism of a new 8K-UHD video microscope [5] that combines 8K-UHD digital camera technology with optical lenses, which we introduced in a previous ‘Letter to the Editor’. As shown in Table 2, there are increasing reports using digital microscope with 4K-3D camera since 2018 [2, 10, 11]. However, video microscopes with conventional 2K-3D cameras are still currently reported [12, 13, 14, 15, 16, 17, 18, 19]. Notably, our report was the first utilizing a video microscope with 8K-UHD camera. To our knowledge, there have not been any other studies using this 8K-UHD camera system.

Table 2

Reports about 2K-3D and 4K-HD digital microscope in recent years.

Camera	Year	Title	Authors	Journal
2K-3D	2020	A high-definition 3D exoscope as an alternative to the operating microscope in spinal microsurgery	Siller S et al.	J Neurosurg Spine. 2020 Jul 10:1–10.
2K-3D	2019	Lessons Learned Using a High-Definition 3-Dimensional Exoscope for Spinal Surgery.	Kwan K et al.	Oper Neurosurg (Hagerstown). 2019 May 1;16(5):619–625.
2K-3D	2018	Initial Experience Using a High-Definition 3-Dimensional Exoscope System for Microneurosurgery.	Sack J et al.	Oper Neurosurg (Hagerstown). 2018 Apr 1;14(4):395–401.
4K-3D	2020	Using a 4K-3D Exoscope for Upper Airway Stimulation Surgery: Proof-of-Concept.	Patel VA et al.	Ann Otol Rhinol Laryngol. 2020 Jul;129(7):695–698.
4K-3D	2020	Free flap microvascular anastomosis in head and neck reconstruction using a 4K three-dimensional exoscope system (VITOM 3D).	De Virgilio A et al.	Int J Oral Maxillofac Surg. 2020 Sep;49(9):1169–1173.
4K-3D	2020	3D Exoscope-Assisted Microvascular Anastomosis: An Evaluation on Latex Vessel Models.	Pinto V et al.	J Clin Med. 2020 Oct 21;9(10):3373.
4K-3D	2020	Use of the ORBEYETM Exoscope in General Surgery: The Advent of Video-Assisted Open Surgery.	Corcione F et al.	Surg Innov. 2020 Oct 15:1553350620965344.
4K-3D	2020	Preliminary clinical experience with the 4K 3-dimensional microvideoscope (VITOM 3D) system for free flap head and neck reconstruction.	De Virgilio A et al.	Head Neck. 2020 Jan;42(1):138–140.
4K-3D	2019	First-in-Man Clinical Experience Using a High-Definition 3-Dimensional Exoscope System for Microneurosurgery.	Khalessi AA et al.	Oper Neurosurg (Hagerstown). 2019 Jun 1;16(6):717–725.
4K-3D	2019	Preliminary Clinical Microneurosurgical Experience With the 4K3-Dimensional Microvideoscope (ORBEYE) System for Microneurological Surgery: Observation Study.	Murai Y et al.	Oper Neurosurg (Hagerstown). 2019 Jun 1;16(6):707–716.
4K-3D	2018	Combined Endoscopic Endonasal and Video-microscopic Transcranial Approach with Preoperative Embolization for a Posterior Pituitary Tumor.	Yoshida K et al.	World Neurosurg. 2018 Nov;119:201–208.

The main advantage of our system is the ultra-high-resolution image quality of 8K-UHD digital images can improve the shadow drop effect and depth of field by changing the optical axis of the lens. Typically, if the position of the camera from which the images are being received is positioned far from the field of view and the field of view is increased through a zoom effect, then, theoretically, the depth of field is deepened. However, image quality is inversely proportionate to distance. By contrast, moving the camera position closer to the field of view will deliver higher image quality, though the depth of field becomes shallower. For microsurgery, which requires dynamic action, this makes procedures such as suturing almost impossible. Therefore, the level which would deliver pixels matching that of the optical microscope was determined. Results indicated that, in the case of image enlargement from a field of view in excess of 60 cm, deterioration could be discriminated at a level of 0.1 mm. When the camera is moved closer and the image input is set at a distance of around 35 cm (normal video microscope level), the 8K image is projected onto the 8K-UHD vision in a monocular 3D system. However, the image quality became that of a 4K-UHD level, and discernment of the 12-0 suture thread degraded. Moreover, a 3-dimensional appearance can be obtained in a variety of ways, but in humans a 3D representation is determined comprehensively within the brain. Humans convert 2D images projected onto the retinas of both eyes into 3D images by the visual physiological mechanism within the brain based on multiple clues (physiological binocular cues and empirical/pictorial monocular cues) to perceives the depth of space and the three-dimensional effect of objects. Therefore, human stereoscopic vision is thought to be a 2.5-dimensional perception, comprised of depth information added to the 2-dimensional vision [9]. The 8K-UHD camera technology can also be used to measure higher-order brain functions. It has already been reported to have been utilized in measuring the brain activity of mice, and that, when using 8K-UHD technology, the neuroactivity in synaptic structures of a size of 0.5 um is 25 times larger than with conventional methods and can be viewed at twice the speed [20]. When the human eye sees the near field with both eyes, the binocular parallax stereoscopic view is the most important [9]. Parallax is determined due to the difference in the position of two points of view. The direction in which the target point is seen differs, or because of the angular difference in parallax, the camera angle introduces a large drop shadow effect with large angular imaging.

Conclusions

In recent years, the clinical application of super-microsurgery has been progressing in accordance with the technological advances in microsurgery [21]. Furthermore, active hands-on uptake is also progressing [22]. With the improvement of ultra-high-quality digital images featuring this type of pseudo-stereoscopic vision, brought about by its clinical application by numerous experts, there is the potential to create a new surgical area that transcends conventional optical microscope images [5].

Declarations

Author contribution statement

Hiromasa Yamashita: Conceived and designed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Eiji Kobayashi: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data.

Funding statement

This work was supported by AIR WATER BIODESIGN INC. (30-148).

Data availability statement

Data included in article/supp. material/referenced in article.

Declaration of interests statement

The authors declare the following conflict of interests: Eiji Kobayashi is a medical advisor of AIR WATER BIODESIGN INC. Hiromasa Yamashita is an employee of AIR WATER BIODESIGN INC. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Additional information

No additional information is available for this paper.