Simultaneous photoacoustic microscopy, spectral-domain optical coherence tomography, and fluorescein microscopy multi-modality retinal imaging.

The goal of this study is to further develop a multi-modality eye imaging system and evaluate its feasibility of acquiring images of different modalities simultaneously. An integrated multimodality imaging system combining spectral-domain optical coherence tomography (SD-OCT), photoacoustic microscopy (PAM), and fluorescence microscopy (FM) was developed, and its performance for eye imaging was validated on multiple clinically-relevant retinal disease models in vivo in rabbits. OCT imaging allows for visualization of the different anatomic retinal layers with high axial resolution. PAM can be used to image vasculature, angiogenesis, and hemorrhages. The leakage of neovascularization can be verified with FM and fluorescein dye. Simultaneous imaging with OCT, PAM, and FM ensures co-registration of the three modalities without being affected by motion artifacts caused by breathing, body or eye movements, and heartbeat. This simultaneous multi-modality eye imaging system could be a new tool for applications both in ophthalmology and other fields.

Introduction

Since the eye is optically transparent and the retina can be easily accessed by light, ophthalmology has a long and rich legacy of benefiting from optical imaging methods for over 150 years [1], including fundus photography, fluorescein angiography (FA) [2], indocyanine green angiography (ICGA) [3], optical coherence tomography (OCT) [4,5], and scanning laser ophthalmoscopy (SLO) [6]. Fundus photography provides a rapid, wide field view of the retina in a single image capture; however, its depth-resolving capability is limited. By demonstrating the leakage of neovascularization, fundus FA remains the gold standard for evaluation and follow-up of neovascular diseases of the retina and choroid, such as proliferative diabetic retinopathy and neovascular age-related macular degeneration. Fundus FA, however, does not provide a 3D image and provides limited visualization of the choroid. Although ICGA can reveal choroidal circulation, it is invasive and requires the administration of an exogenous contrast agent. Obtaining a high resolution, three-dimensional retinal image with fluorescence imaging is challenging. OCT is able to image retinal morphology and retinal thickness by providing cross sectional and 3D anatomic images of the retina with high resolution. OCT angiography (OCTA) provides volumetric angiography image with the ability to demonstrate the blood flow information [7]. Both OCT and OCTA are limited by a relatively small field of view, inability to show leakage, limited view of microaneurysms, and image artifacts. Although SLO can capture almost the entire retina in one image, there is a trade-off between the spatial resolution and the field of view [8]. As a novel biomedical imaging method, photoacoustic microscopy (PAM) has a unique capability to non-invasively map the optical absorption properties in deep biological tissues with high spatial resolution, which can potentially compensate the limitations of current ocular imaging techniques [[9], [10], [11]].

Multi-modal retinal imaging is defined as the use of more than one complementary technological system that is used to acquire images, concurrently or in a short period of time, for the purpose of diagnosis, prognostication, management, and monitoring of disease [[12], [13], [14], [15]]. It takes the merits of the different modalities and compensates for their limitations, which will be highly beneficial to ophthalmology [16,17]. Current multi-modal retinal imaging performs each modality imaging sequentially and performs post-processing image registration given the limited eye fixation time. Although this method can provide the multi-modality information, it is limited by the eye fixation time, rapid eye saccades, and body motion which can increase the difficulty of performing image registration and increase image artifacts [18]. Different algorithms have been proposed to perform image fusion with the different modalities; the image stretching and warping will induce additional artifacts and uncertainties for diagnosis [19,20].

Although our previous study had developed an integrated multi-modality imaging system, due to sharing the same laser system for different modalities, the three different modalities needed to perform sequentially. Since photoacoustic microscopy (PAM) and FM shared the same optical path, the system needed to be adjusted to avoid interference when shifted to different modalities [21]. The previous imaging system took significant time to acquire images and thus had distortions and artifacts caused by body and eye motions. With sequential imaging in the previous system, it was also difficult to perform image fusion and combine the advantages of different modalities. The goal of this study was to further develop a simultaneous multi-modality eye imaging system and explore the feasibility of performing simultaneous PAM, OCT, and FM imaging in vivo. These three imaging modalities were selected due to their unique and complementary nature to provide significant anatomic and functional eye information. An integrated multi-modality imaging system was developed, and the capability to simultaneously acquire images from the three different modalities was investigated.

Method

System setup

Fig. 1 shows the experimental setup for simultaneous multi-modality retinal imaging. The details regarding our previous multi-modality imaging system can be found in our previous papers [21,22]. The system was significantly revised and upgraded so that simultaneous multi-modality imaging became possible. The Q-switched diode pumped solid state (DPSS) laser (SPOT-10-532, Elforlight, UK) for PAM works at a wavelength of 532 nm, and at pulse repetition rate (PRR) up to 30 kHz. The green light from the DPSS laser was coupled into a 3-meter polarization-maintaining single-mode fiber (PM-SMF) through a fiber collimator [23]. The Raman shift of the wavelength in the PM-SMF tuned the 532-nm light to a longer wavelength, as shown by the Raman output in Fig. 2. After passing through a dichroic mirror (DM1 in Fig.1, FF556-SDi01-25 × 36, Semrock), all the wavelengths below 556 nm were removed to avoid interference with fluorescence imaging. The remaining light, as shown by the PAM illumination in Fig. 2, was then used as the light for PAM. The OPO laser (NT-242, Ekspla, Vilnius, Lithuania) was used as the illumination source for FM. With a tunable wavelength from 405 nm to 2600 nm, the OPO laser based FM is compatible with numerous fluorescent dyes. The illumination sources for PAM and FM were coaxially aligned through the dichroic mirror (DM1 in Fig. 1).

Fig. 1

The experimental setup for simultaneous multi-modality retina imaging with integrated spectral-domain OCT (SD-OCT), PAM, and FM. (FC: fiber collimator, DM: dichroic mirror, DCG: dispersion compensation glass).

Fig. 2

Raman shift wavelength measured by spectrum photometer. The generated stimulated Raman scattering (SRS) peak, the excitation wavelength, and the spectrum of PAM illumination source.

A triple-edge standard epi-fluorescence dichroic beam splitter (DM2 in Fig. 1, FF395/495/610-Di01, Semrock) placed before integration with the OCT light was used to couple the excitation lights of PAM and FM. With the wavelength from 795 nm to 1005 nm, OCT illumination light (Ganymede-II-HR, Thorlabs) was coaxially aligned with PAM and FM excitation light before the galvanometer through the third dichroic mirror (DM3 in Fig. 1, FF775-Di01-25 × 36, Semrock). Here, the light beams from different imaging modalities were coaxially aligned to ensure co-registration of the multi-modality images. Sharing the same galvanometer, the excitation lights of different modalities were delivered and focused on the same area of the retina through a telescope configuration.

Both the emission light for FM and the reflection light for OCT travelled back to the telescope configuration and galvanometer. The OCT reflection light from the sample directly went through the third dichroic mirror (i.e. DM3), and combined with the reference light from the reference arm to provide interference, which was detected with the OCT detection system with up to 35-kHz repetition rate. The FM emission light was reflected by the third dichroic mirror (i.e. DM3) and directly went through the triple-edge standard epi-fluorescence dichroic beam splitter (i.e. DM2). After passing through the fluorescence filter, it was collected by an avalanche photodiode (APD) and then digitized by the DAQ card (PX1500–4, Signatec Inc, Newport Beach, CA) with a 300-MHz sampling rate. The acoustic wave induced by PAM illumination light was acquired by a custom-built needle-shaped ultrasound transducer with central frequency of 30 MHz (Optosonic Inc., Arcadia, CA, USA). The detected signal was amplified by a 57-dB low-noise amplifier (AU-1647, L3 Narda-MITEQ, NY) before digitization. Simultaneously, the laser output energy for both FM and PAM illumination was acquired by a photodiode (PD) and digitized using the same DAQ Card with the same sampling rate. Both the PAM laser system and OCT system were working in external mode. A four-channel delay generator (DG535, Stanford Research Systems) triggered by the synchronization signal from the OPO laser with a 1 kHz pulse repetition rate was used to precisely trigger the SPOT laser, the OCT system, the galvanometer, and the DAQ card. With a scanning area of 256 × 256 points, it took about 68 s to obtain the images from the three modalities at once.

The lateral resolutions of the PAM and the SD-OCT were previously quantified to be 4.1 μm and 3.8 μm, respectively; whereas the quantified axial resolutions of the PAM and the SD-OCT were 37.0 μm and 4.0 μm, respectively [21,22]. In this study, a continuous wave (CW) laser with a central wavelength of 900 nm and energy of 0.95 mW in front of the cornea was applied for the OCT. A laser wavelength of 556−620 nm and energy of 80 nJ per pulse before the eye were used for PAM. A laser wavelength of 480 nm and energy of 2 nJ per pulse were utilized for FM. According to the ANSI safety limit for ocular exposure, the laser energy used for PAM and FM should not exceed 160 nJ, while the laser energy for OCT should be less than 1mW [24]. All three different modalities in this study were working below the ANSI safety limits.

3D image fusion

3D image fusion was performed online by using the simultaneous multi-modality imaging data. The images from the three modalities were imported to Amira to perform image fusion. Due to coaxially aligned illumination lights for different modalities, the XY planes of the three modalities were naturally coregistered. A 3D fusion image was obtained by simply adjusting the Z-axial position of each modality, where the Z-axial of the OCT image was regarded as the gold standard for its high axial resolution. In a 3D fusion image, the OCT image and the PAM angiography image were combined in 3D, while the 2D FM image was placed on the top of the fusion image.

Animal preparation

All the experimental procedures were performed in accordance with the ARVO (The Association for Research in Vision and Ophthalmology) Statement for the Use of Animals in Ophthalmic and Vision Research and were approved by the Institutional Animal Care & Use Committee (IACUC) of the University of Michigan (Protocol PRO00008566, Photoacoustic & Molecular Imaging of the Eye).

Six New Zealand rabbits (both genders, 2–4 months, 2.0–3.0 kg) were involved in this study. The rabbits were anesthetized with a mixed solution of ketamine (40 mg kg−1) and xylazine (5 mg kg−1) by intramuscular (IM) injection. The pupils of the eyes were dilated with 2.5 % phenylephrine hydrochloride and 1 % tropicamide ophthalmic solution. Topical anesthesia was used by 0.5 % topical tetracaine drops prior to initiation of the experiments. A vaporized isoflurane anesthetic (1.5 % isoflurane) (Surgivet, MN, USA) and a V-Gel® (D10004, Jorgensen Laboratories, Loveland, CO) were used to maintain anesthesia. A V8400D Capnograph & SpO2 Digital Pulse Oximetry (MWI Animal Health, Boise, ID) was utilized to evaluate anesthesia level and continuous monitoring of the heart rate and respiratory rate. Rectal temperature was measured every 15 min. A water-circulating heating pad (TP-700, Stryker Corporation, Kalamazoo, MI) was used to keep the body temperature stable. To avoid corneal dehydration and ensure acoustic coupling to the ultrasound transducer, balanced salt solution (BSS, Altaire Pharmaceuticals, Inc., Aquebogue, NY) was applied liberally to the eye surface. For the FM imaging, fluorescein sodium (10 %, 0.1 mL kg−1, Akorn Inc, Lake Forest, IL) was intravenously administrated through the marginal ear vein.

To evaluate the performance of the proposed system, rabbits with different clinically relevant retinal disease models were involved in this study, including retinal detachment, and retinal vein occlusion (RVO) induced choroidal neovascularization (CNV). To create the rabbit RVO, Rose Bengal (Sigma, St. Louis, MO) with concentration of 50 mg/mL was administrated intravenously through the marginal ear vein with a sustained-release injection with a total volume of 3 mL [25,26]. During the injection, an argon green laser light (Vitra 532 nm, Quantel Medical, Cournon d’Auvergne, France) was used to treat the main retinal veins with 10 shots for each vein (150 mW, 75 μm, 500 ms) until the blood vessel was completely occluded and the blood flow was stopped. About one month after creating the RVO, CNV developed. Retinal detachment was induced by intravitreal injection of DL-α-aminoadipic acid (DL-AAA), leading to chronic retinal neovascularization. Wild type New Zealand white rabbits received a single intravitreal injection in one eye of 50 μL of 0.025 M DL-AAA [27]. Fundus photography and FA were used to follow the changes in the rabbit retina until the pathological changes occur.

Result

Normal retinal blood vessel

The results in Fig. 3 show the images from the three different modalities of a rabbit with normal retinal and choroidal blood vessels. As shown in Fig. 3(A), the 3D structure of whole retina can be obtained through OCT en-face image. The different layers of the retina can be clearly distinguished on the OCT B-scan image in Fig. 3(B). In the PAM image shown in Fig. 3(C), the high resolution angiography image shows the distribution of the retinal and choroidal blood vessels. In Fig. 3(D), the circulation of the fluorescein dye is presented by the FM image.

Fig. 3

Simultaneous multi-modality images of the retina in a normal rabbit eye. (A) The side view of 3D OCT image; (B) The 2D cross-sectional view of OCT image; (C) The peak-value projection of PAM image; (D) The 2D FM image.

Retinal detachment

Retinal detachment occurred 2 weeks after the intravitreal injection of DL-AAA. Simultaneous multi-modality imaging was performed after the retinal detachment developed. As shown in the OCT images in Fig. 4(A) and (B), the retinal detachment can be clearly distinguished. OCT imaging precisely indicates the subretinal fluid between the neurosensory retina and the retinal pigment epithelium (RPE) layer, and provides quantifiable volumetric data. In Fig. 4(C), the PAM image shows the vasculature of the retina and choroid. Due to the high contrast provided by hemoglobin, the hemorrhage can be distinguished clearly in the PAM image. In the FM image shown in Fig. 4(D), part of the fluorescein signals from choroidal blood vessel was blocked by the hemorrhage. Although PAM can display differences between the normal retina and hemorrhage, it provides limited information on the retinal detachment. Fluorescein angiography provides the vasculature circulation information and provides limited information regarding the retinal detachment.

Fig. 4

Simultaneous multi-modality images of retinal detachment in the rabbit eye. (A) The side view of 3D OCT image; (B) The 2D cross-sectional view of OCT image; (C) The peak-value projection of PAM image; (D) The 2D FM image.

Retinal vein occlusion complicated by choroidal neovascularization

Images of rabbit eyes with CNV were taken 5 weeks after the creation of the RVO model. As shown in Fig. 5(A) and (B), the OCT images demonstrate significant retinal atrophy, with thinning of the neurosensory retina particularly the inner retina. The vasculature is hard to distinguish on the OCT images. Although we still can see the proliferative membranes above the retina, the retinal blood vessels are absent. In the PAM image shown in Fig. 5(C), the newly generated choroidal vasculature can be clearly seen. The spatially distributed retinal vessels located at the lower side of the CNV can also be seen. The difference in characteristics between these two kinds of vasculatures cannot be distinguished easily by the PAM image. The patterns of hyperfluorescence and stereoscopic FM images yield valuable information about the leakage of fluorescein dye from retinal and choroidal vessels vascular endothelial cells or from neovascularization. In Fig. 5(D), although the FM cannot provide a high resolution and high sensitivity image of vasculature in rabbit retina, the leakage property of the CNV area was clearly demonstrated with FA. With the information from FM and PAM angiography images, both the high-resolution structure information and leakage property of the vasculature in rabbit eye can be obtained.

Fig. 5

Simultaneous multi-modality images of CNV and RNV in rabbit eye. (A) The side view of 3D OCT image; (B) The 2D cross-sectional view of OCT image; (C) The peak-value projection of PAM image; (D) The 2D FM image. The white dashed area indicates the region of fluorescein leakage corresponding to the neovascularization.

Fig. 6 shows 3D image fusion of the images from the three different modalities. As the images from the three modalities were acquired simultaneously via a single scan, the image fusion can be achieved simply by adjusting the Z-axial of each modality. The integrated 3D fusion image shows the retina structure from OCT in gray scale, PAM angiography information with red color, and FM information in green color. The fusion image is excellent in presenting the 3D multi-layer of retina via OCT, 3D angiography information via PAM, and 2D circulation of fluorescein dye via FM.

Fig. 6

3D fusion image of simultaneous multi-modality imaging. (A)-(D) Multi-angle view of 3D fusion image from a normal rabbit retina. The red color represents the PAM image, the gray color indicates the OCT image, and the green color shows the FM image.

Discussion and conclusion

This study describes the first fully-integrated simultaneous multi-modality imaging system combining OCT, PAM, and FM which was tested in vivo in clinically-relevant rabbit eye models. Compared with the previous sequential multi-modality imaging system, this system is able to achieve simultaneous multi-modality imaging. To avoid the interference of illumination of different modalities, the wavelength of PAM illumination light was shifted to above 556 nm, which was beyond both the excitation spectrum and emission spectrum of the fluorescein dye used. The OCT was performed in the near-infrared window with CW light, and its wavelength was far away from the PAM and FM illumination lights. Three different lights were combined together through dichroic mirrors. With the illumination lights from the three different modalities aligned coaxially before the scan head and sharing the same galvanometer system, the X–Y plane of the three modalities images were naturally coregistered. Compared with sequential multi-modality imaging which has to involve image registration by using image stretching and image warping to eliminate the misalignment, this simultaneous multi-modality imaging system can acquire images from different modalities at once via a single scan, which is not only faster but also reduces the motion artifacts. By simply adjust the Z-axial positions, the images from the different modalities can be registered easily in 3D.

Multi-modality imaging provides unique advantages to visualize anatomic and functional information of pathologic conditions. OCT allows for excellent visualization of the different retinal layers with high axial resolution. Due to the lower scattering contrast between the retinal layer and neovascularization, OCT, however, is unable to distinguish the small neovascularization, especially when retinal atrophy is present. Although OCTA can provide high resolution angiography imaging, it cannot provide the information of hemorrhage without blood flow or in slow blood flow situations like microaneurysms. Based on the optical absorption contrast between tissues, PAM can selectively image blood vessels of the retina and choroid as well as bleeding with a larger penetration depth than OCT. Using PAM, the high resolution and high sensitivity angiography can be achieved even with retinal atrophy, retinal detachment, and preretinal fibrovascular membranes. FM adds additional information by demonstrating the leakage of neovascularization with fluorescein dye, which is the gold standard to validate neovascularization in retina clinics. By performing simultaneous multi-modality imaging, the images from the three modalities can be easily fused into a single 3D image. In the resulting fusion image, the location of the vasculature and their leakage properties can be directly visualized. OCT, PAM, and FM all give unique anatomic and functional information which complement one another to provide comprehensive diagnostic information of the retinal state and function.

Although the current study involves spectral domain OCT imaging, OCTA can also be integrated in the simultaneous multi-modality imaging system. OCTA acquires the variation in OCT signal caused by moving particles through multiple B-scans in the same location. Since all the modalities can work in the external trigger mode, integrated OCTA can be easily achieved by precisely controlling the timing sequence of the different modalities. Meanwhile, functional PAM with oxygen saturation and blood flow measurements can also be integrated into this system. Photoacoustic oxygen saturation measurement, by utilizing the spectroscopic differences between oxygenated and deoxygenated hemoglobin, is capable of quantitatively measuring the blood oxygen saturation. To achieve this function, the PAM illumination source in the current system will be replaced by the light at two or more wavelengths which will pulse alternatively.

In conclusion, multi-modality imaging can combine the merits and compensate for the limitations of individual modalities to give additional information that cannot be acquired by any of the modalities alone. Simultaneous imaging with OCT, PAM, and FM ensures co-registration of the three modalities without being affected by eye motion and saccades. The multi-modality eye imaging system demonstrated in this work could be developed into a new tool for ophthalmic applications.