Advanced multispectral image-processing endoscopy system for visualizing two-dimensional hemoglobin saturation and relative hemoglobin concentration.

Background and study aims  The association of tumor hypoxia and vascularization with malignant progression is recognized, and detection by measuring tissue hemoglobin (Hb) saturation and concentration has attracted attention. In this study, we designed a simple algorithm and multispectral image-processing endoscopy system to map relative Hb concentration and Hb saturation for detection of tumors in small animal viscera in vivo. Materials and methods  We designed and validated an optical filter-equipped endoscope system for two-dimensional visualization of Hb concentration and saturation maps and used it in a real-time video examination. A simplified method based on spectral data capture and analysis of defuse reflection of mucosa, including image capture and data processing of the spectral features of Hb oxygenation, was developed. Results  An Hb saturation calibration curve was obtained. Then, differences in oxygenation levels between normal mucosa and in vivo tumors in a small animal model were determined by using the new method and endoscope system. Conclusions  A multispectral image-processing endoscopic system with a mapping frame rate comparable to that of white light imaging systems (7.5 frames/second) was developed.

Introduction

Image-enhanced endoscopy is widely used for detection and diagnosis of diseases. The principle of some diagnostic methods is recognition and classification of images, which is based on an individual's experience. Filtered images are combined by using image-processing techniques, which markedly improve visualization of lesions 1 . However, image-enhanced endoscopic detection and diagnosis of disease is still based on each operator’s subjective judgment.

Recently, quantitative diffuse reflection spectroscopy for gastrointestinal examination was developed for detection of noninvasive disease diagnosis 2 . The system uses hyperspectral imaging to successfully capture biological information 3 . We previously reported that a large part of the diffuse reflection data of the normal mucosa obtained by using this system were dominated by blood and scatter signals. Furthermore, two-dimensional (2 D) mapping of relative hemoglobin (Hb) concentration and Hb saturation mapping was also performed, which suggested its value for medical use 4 .

Relative Hb concentration and Hb saturation are useful for detection and diagnosis of gastrointestinal disease. These data provide quantitative and objective information relevant to biological activity. Associations of tumor hypoxia and vascularization with malignant progression have been reported previously 5 . Despite reports of point and 2 D mapping of this information based on hyperspectral data, there is no system as yet that provides real-time saturation video imaging for practical use. Furthermore, the value of the information for clinical use, such as for detection and diagnosis of lesions, is unclear.

Conventional hyperspectral data capture and analysis procedures are not always optimal for use in a practical commercial system for assessing relative Hb concentration and Hb saturation because of difficulty with the amount of data that can be captured, difficulty ensuring light intensity, and complexity of postprocessing. In the current study, we designed a simple process and algorithm to obtain Hb saturation and relative Hb concentration. By capturing featured wavelength area data and reducing the calculation load, we achieved a frame rate of 7.5 frames per second. We then designed and prototyped an endoscopy system using this method that provided real-time relative Hb concentration and Hb saturation mapping functionality. This simplified and practical system was applied to relative Hb concentration and Hb saturation mapping of tumors in small animal viscera in vivo.

Material and methods

Principle of the optical filter system

The Hb spectrum change during the oxy-Hb to deoxy-Hb transition is shown in Fig. 1 . The basic spectra for saturation of 100 % oxygenated and 100 % deoxygenated Hb are shown, and the spectral wavelength region from 524 to 582 nm is highlighted.

Fig. 1

 Typical spectra of the oxy-hemoglobin (Hb; red) and deoxy-Hb (blue) spectrum 1.0 × 10−4 g/L. Absorption data were derived from “Absorption of Hemoglobin by Scott Prahl, Oregon Medical Laser Center.” https://omlc.org/spectra/hemoglobin/

An inverted change in transmittance between the isosbestic points, which were derived from the oxy-deoxy transition, is observed, and a marked transmittance change between 524 and 582 nm was selected for this system. Measuring changes in the spectra corresponds to change in the quantitative ratio of oxy-Hb and deoxy-Hb and can be analyzed by using the spectral analysis method. The Hb saturation is defined by Eq. 1.

Single xenon light source and optical filter system were developed for our Hb concentration and saturation mapping endoscope system, which is different from the previously reported saturation mapping system in the concept of optical design 6 . The system has been certified by PMDA JAPAN (The Pharmaceuticals and Medical Devices Agency; https://www.pmda.go.jp/english/symposia/0147.html) ( Fig. 2 ). Red – green – blue image signals and narrow (544 to 570 nm), wide-filtered (524 to 582 nm) image signals were sequentially collected. By calculating the ratio of the narrow-filtered signal to the wide-filtered signal of each pixel, we can obtain the Hb saturation related value. The relationship of the ratio of the narrow and wide filters with Hb saturation is shown in Fig. 3a and Fig. 3b . Using the values of the red, green, and blue signals and wide-filtered signals, the Hb concentration was calculated without the influence of saturation. The Hb saturation value was decided by using the ratio of the narrow-filtered signal to the wide-filtered signal versus the Hb concentration curves.

Fig. 2

 Whole image of endoscopy system.

Fig. 3 a

Simulated Hb saturation versus narrow-filter area spectral data of Hb for different Hb concentrations. b Simulated Hb saturation versus wide-filter area spectral data of Hb for different Hb concentrations. The wide-filter data are independent of Hb saturation and dependent on Hb concentration.

Calibration of instruments for quantitative analysis

Calibration curves for Hb oxygen saturation of the “spectrometer” versus “endoscope” were prepared by using the experimental arrangement in Fig. 4a and least-squares method data analysis shown in Fig. 4b . The Hb saturation level of the blood solvent was controlled by a reducing agent, and each saturation condition was measured by the two instruments.

Fig. 4 a

Experimental arrangement of Hb saturation calibration. b Calibration curve of Hb saturation captured by the developed endoscope system. Spectral data and curve-fitting analysis used to obtain y-axis versus standard data (x-axis).

As an experimental specimen, a 3 × 10 −3 g/L blood solution was prepared in a vessel. The path length of the solution was 10 mm. The Hb oxygenation level was controlled using Na 2 S 2 O 4 7 . A filtered image and a red-green-blue video image were captured by the endoscope. Additionally, spectral data from 400 to 800 nm at 1-nm resolution were also collected by using a single-fiber spectrometer HR2000 (Ocean Optics) nearly at the same time. The Hb saturation value for each instrument was calculated. A single-fiber spectrometer (HR2000) detected incident light through the fiber at the measuring point. The endoscope detected 968 × 496 points as image data, and the average of the center 100 points area was used for the plot ( Fig. 4b ).

Experimental observation of a tumor in vivo

A small animal model of direct tenuous implantation of HLaC-79 human squamous cancer cells was prepared as an observation model. Tumor growth was investigated using an in vivo imaging system (IVIS Spectrum PerkinElmer Inc., Waltham, Massachusetts, United States) prior to dissection. Relative Hb concentration mapping and Hb saturation mapping for the in vivo tumor were performed by using our system. The Hb concentration and saturation combined image was also acquired. The image combination technique uses a mask-based Hb concentration distribution. We can select the concentration scale and range depending on the individual's difference.

All animal care and experimental protocols were approved by the Animal Ethics Committee of the Kyushu University Faculty of Medicine and were performed according to the recommendations of the Committee for Care and Use of Laboratory Animals, Kyushu University Faculty of Medicine. Male C57BL/6 mice, aged 5 weeks, were purchased from Charles River Laboratories Japan, Inc. (Yokohama, Japan) and maintained in temperature- and light-controlled chambers (24 °C, 12 h/12 h light-dark cycle). Prior to the experiments, all animals were fed a normal diet for 1 week for acclimation and had free access to tap water and appropriate food (MF diet; Oriental Yeast Co., Tokyo, Japan).

Results

Calibration curve

For evaluation of the function of our system, we obtained a calibration curve and performed a correlation with the actual saturation values by using the Hb saturation phantom ( Fig. 4b ). The Hb saturation calibration curve was obtained by using stable oxygenation level samples in transparent containers. The standard error of the predicted y-value for each x in the regression was 4.8 %.

Experimental observation of in vivo tumor

Hb saturation mappings of the small animal ligation model are shown in Fig. 5a and Fig. 5b . A decrease in oxygen saturation due to vascular ligation has been indicated in green.

Fig. 5 a

Experimental vessel ligation model. Red, green, and blue video endoscope image of the mouse viscera. The ligation point is indicated by an arrow. b Hb saturation map of the ligation model mouse by the endoscope system. Lower hemoglobin saturation area is indicated in green.

The small animal cancer model is shown in Fig. 6a , Fig.6b , Fig. 6c , and Fig. 6 d . The red, green, and blue video endoscope image and saturation map and Hb concentration-masked Hb saturation map are shown in Fig. 6d . The malignant disease area of the high Hb concentration and the low Hb saturation is enhanced by this procedure.

Fig. 6 a

Experimental cancer model. Red-green-blue video endoscope image of the mouse viscera. The tumor area is indicated by an arrow. b Hb saturation map of the tumor generated by the endoscope system. c Hb concentration map of the tumor generated by the endoscope system. d Hb concentration-masked Hb saturation map. Higher Hb concentration area was enhanced by an Hb concentration mask. Higher Hb concentration and lower Hb saturation area, influenced by the tumor, were observed as a blue area.

Discussion

As suggested in previous studies, quantitative assessment based on spectroscopy of Hb concentration and Hb saturation is one of the latest developments in real-time endoscopy field 5 . We developed a concise and practical method based on this concept and tested its functionality for current endoscope systems in clinical use

Optical design and analysis

Spectroscopy approaches for measurement of typical Hb concentration and Hb saturation have been previously reported 5 6 8 . In the current study, we developed and validated an Hb concentration and saturation mapping system for endoscopy. The system provides red-green-blue video movie imaging and time-sequential Hb concentration and saturation map.

We selected an optical filter to integrate the intensity and simplify the postprocessing procedure. The cut-on and cutoff wavelengths of the two optical filters were designed to match the isosbestic points of Hb because the influence of cut-on and cutoff wavelength errors in the filters are smaller than those of the center area of the peak ( Fig. 1 ).

It is also important for spectral band selection to consider the influence of scatter. Scatter in the defuse reflection data of the mucosa was previously reported in numerous studies 9 .

The scatter influence of mucosa is serious in the wavelength area of < 500 nm, and the Hb saturation signal weakens as scatter level increases. For this reason, a > 500 nm area spectral peak was selected. Furthermore, for calculation of the saturation related value, the spectral baseline of the narrow-filtered signal (544 – 570 nm) and the wide-filtered signal (524 – 580nm) were overlapped, and the influence of scatter in the reflection spectra was minimized. The overlapped optical baseline helped generate rational and reliable calculation results.

We developed a simplified calculation procedure based on (Equation 1, Fig. 3 ) and a combination technique for obtaining an Hb concentration-masked Hb saturation map. This technique helped us to recognize an Hb concentration and saturation area in one figure ( Fig. 6d ). In addition, the vascular pattern and its saturation value can be recognized effectively with this method. This practical use of the imaging method loading in this system should be indispensable for clinical research and applications. The mapping frame rate of this system is comparable to those of white-light image systems; however, to prevent artifacts, which are caused by image shift and deformation of the subject itself, a higher frame rate is preferable (> 15 frames/sec), and improvement in capture rate is necessary.

Future clinical applications

Measurement of relative Hb concentration and Hb saturation mapping is not limited to endoscopy and can be used to monitor blood flow recovery and tissue oxygen supply after surgery ( Fig. 5b ). In addition to providing quantitative assessment of blood status, these values can be used to detect spatial abnormalities in tissues. Fusion analysis using scientific instruments is also effective for increasing accuracy of diagnosis. Overall, we suggest that future development of medical equipment use spectral imaging and analysis. For example, 2 D Hb saturation surgical system, which can indicate tissue activity, can be used. We hope that the results of this study will provide new criteria for detection and diagnosis using endoscopy of various diseases and to improve gastroenterology clinical outcomes 10 .

Conclusion

In this study, we designed and validated an optical filter-equipped endoscope system that enables 2 D visualization of Hb concentration and Hb saturation maps. Finally, differences in oxygenation levels between normal mucosa and those of in vivo tumors in a small animal model were determined by using our new endoscope system.