Snapshot hyperspectral retinal camera with the Image Mapping Spectrometer (IMS).

We present a snapshot hyperspectral retinal camera with the Image Mapping Spectrometer (IMS) for eye imaging applications. The resulting system is capable of simultaneously acquiring 48 spectral channel images in the range 470 nm-650 nm with frame rate at 5.2 fps. The spatial sampling of each measured spectral scene is 350 × 350 pixels. The advantages of this snapshot device are elimination of the eye motion artifacts and pixel misregistration problems in traditional scanning-based hyperspectral retinal cameras, and real-time imaging of oxygen saturation dynamics with sub-second temporal resolution. The spectral imaging performance is demonstrated in a human retinal imaging experiment in vivo. The absorption spectral signatures of oxy-hemoglobin and macular pigments were successfully acquired by using this device. 2011 Optical Society of America

1. Introduction

Hyperspectral retinal imaging is a novel technique for non-invasive ocular diagnosis [1,2]. Instead of only providing intensity images, the hyperspectral retinal camera is capable of acquiring both spatial and spectral retinal information and constructing an (x, y, λ) 3D datacube for multivariable data analysis [3,4]. Since light reflected from retinal structures (e. g., the macula) often have unique spectral signatures which are commonly considered to be associated with specific ocular diseases [5], the capability of identifying these spectral signatures in-vivo will be of high value in clinical diagnosis and treatment. However, most traditional hyperspectral retinal cameras are scanning-based systems—they either scan in the spatial domain, e.g. push-broom slit scanning spectrometers [1], or scan in the spectral domain, e.g., liquid-crystal-tunable-filters [6] and sequential illuminated multicolor LEDs [7]. Incorporation of scanning-based hyperspectral imagers in retinal imaging causes severe motion artifacts and pixel misregistration problems because the human eye is constantly moving. To overcome this limitation, recently a snapshot hyperspectral retinal camera which utilizes a computed tomographic imaging spectrometer (CTIS) has been developed for fundus imaging [8]. Although this CTIS-based hyperspectral retinal camera can acquire multiple spectral scenes within a single integration event, the CTIS technique itself requires extensive computational cost and is limited by resolution constraints [9].

In this article, we present a novel snapshot hyperspectral retinal camera which uses an Image Mapping Spectrometer (IMS) [10-12] as its spectrum detector. The IMS is a parallel acquisition hyperspectral imager that can capture hyperspectral datacubes without scanning. It also allows full light throughput across the whole spectral collection range due to its snapshot operating format. By integrating the IMS with a traditional retinal camera (Topcon TRC50X, Topcon Inc, Tokyo, Japan), we have achieved simultaneous 48 spectral channel imaging of the human retina in vivo. The reflectance and absorption spectra from vessels and the macula were measured, from which the known spectral absorption signatures of oxy-hemoglobin and macular pigment were successfully identified. The previously unknown spectral reflectance of an optic disc druse was also measured.

2. System description

The system setup of the hyperspectral retinal camera is shown in Fig. 1 (a) . The IMS is coupled to the back image port of the Topcon TRC 50X retinal camera to acquire (x, y, λ) datacubes. The optical components of the retinal camera and the IMS are shown in Fig. 1 (b). The illumination is provided by the tungsten light of retinal camera. After being collimated, the illumination light is reflected by an annular mirror and then focused on to the eye’s pupil, creating a uniformly illuminated field on the retina. The reflected light from the fundus is collected by retinal camera’s front lens and forms an intermediate image at its back image port, where the entrance port of the IMS is co-located.

Fig. 1

Snapshot hyperspectral retinal camera with the IMS. The IMS is coupled to the back image port of a traditional retinal camera. The optical layout inside the IMS is detailed in [12].

The operation of the IMS is based on the image mapping principle, which has been detailed elsewhere [10,11]. Briefly, the intermediate image at the entrance port is first reimaged onto a custom-fabricated component—the image mapper. The image mapper consists of hundreds of mirror facets, and each mirror facet is around 70 µm wide and has a two-dimensional tilt [13]. The image mapper cuts the intermediate image into strips and reflects them to different locations of a CCD camera. Due to tilt angle differences of mirror facets on the image mapper, blank regions are created between adjacent image strips at the detector plane. The strips of reflected light from the image mapper are then dispersed by a prism array and reimaged onto their associated blank regions by an array of reimaging lenses. In this way, each pixel on the CCD camera is encoded with unique spatial and spectral information from the sample. By applying a simple image remapping algorithm [11], a hyperspectral (x, y, λ) datacube is acquired. Since no scanning is employed, the IMS features high optical throughput and datacube acquisition rate (currently up to 7.2 fps [12]). The spatial sampling of each acquired spectral scene is 350 × 350 pixels, and the measured spectral range is from 470 nm to 650 nm with ~4 nm bandwidth.

3. Hyperspectral imaging of the retina in vivo

To demonstrate the spectral imaging performance of the IMS-based hyperspectral retinal camera, the retina of a 25 year old healthy female volunteer with a known ocular diagnosis of optic disc drusen [14] was imaged in vivo. The subject’s pupil was dilated with mydriatic (Mydriacyl, 1%) 15 minutes before the experiment. The hyperspectral retinal camera was working in the reflectance imaging mode, in which the internal light source of the retinal camera provided constant illumination of the structures. The retinal images were taken at the 50 degree viewing angle of the camera, of which an approximately 20 degree retinal image was captured by the optics of the IMS. The IMS was operated at 5.2 fps with ~180 ms integration time for each frame. The acquired (x, y, λ) datacube is displayed as a panchromatic image in Fig. 2 (a) . The coloration of each pixel is converted from corresponding spectral data with a specific algorithm [15]. Selected images from a total of 48 spectral channels are shown in Fig. 2 (b) (see a scan of all acquired wavelengths in Media 1). Note that vertical stripes show up in Media 1. These stripes are image artifacts and are caused by the mirror facets’ reflectivity variations in the current IMS. In order to provide a baseline reference, a retinal camera image without the IMS attached is also captured and shown in Fig. 2(c). The image resolution of the standard fundus photo is about the same as the hyperspectral composite.

Fig. 2

Hyperspectral imaging of the retina centered at the optic disc in vivo. An optic disc drusen is seen at about the one o’clock position at the edge of the disc. (a) Panchromatic image display of acquired (x, y, λ) datacube. The coloration of each pixel is converted from corresponding spectral data. (b) Selected images from a total of 48 spectral channels. A scan of all acquired wavelengths is shown in Media 1. Note the greater spectral reflectance of the optic disc druse in the 530 nm to 580 nm wavelengths. There is also an atypical retinal vessel branching pattern, which often accompanies optic disc drusen. (c) Baseline reference image captured without the IMS attached.

The ability to visualize oxygen delivery into the eye in vivo is important in clinical studies because it would aid our understanding of the way that oxygen is provided and used in the eye, both in healthy and diseased conditions. Since the oxygenation can be estimated from blood oxy-hemoglobin concentration [1], we first measured the absorption spectra from a retinal arteriole on the optic nerve (see Fig. 3 ) to test the feasibility of the proposed system in recovering oxy-hemoglobin’s absorption spectral signature. The IMS was spectrally calibrated with respect to a standard light source (Ocean optics, PN: LS-1-CAL-INT) before the experiment. The spectral range from 510 nm to 586 nm was chosen for analysis because within this range the blood absorption has a major contribution to the overall spectral reflection while ocular media absorption and scattering by erythrocytes are minimal [4,16]. The reflectance spectrum S (Fig. 3(b)) at the vessel was acquired by averaging over pixels’ spectra in the circled area. In order to acquire the absorption spectrum in the same area, the lamp’s illumination spectrum S was first measured by placing a white paper in front of the retinal camera’s front lens and averaging over pixels’ reflectance spectra in the field of view. Then the absorption spectrum S (Fig. 3(c)) was calculated by subtracting the normalized reflectance spectrum S from the normalized lamp’s illumination spectrum S in the spectral range 510 nm to 586 nm, e.g.

Fig. 3

Measured (b) reflectance spectrum and (c) absorption spectrum of oxy-hemoglobin in a retinal arteriole on the optic nerve; measured (d) reflectance spectra of druse and normal disc substance and (e) their reflectance ratio vs. wavelength

The two dominant peaks in the resulting spectrum (Fig. 3(c)) correspond to the peak absorptions of oxy-hemoglobin at around 540 nm and 570 nm, as expected from an arteriole with nearly completely O2 saturated hemoglobin [1]. Additionally, since our subject carries a diagnosis of optic disc drusen (seen at position B in Fig. 3 (a)), we also measured the reflectance spectrum of one druse relative to the reflectance of normal disc substance (position C in Fig. 3(a)). The measured reflectance spectra of the druse and normal disc substance are shown in Fig. 3(d) and their ratio vs. wavelength is calculated and shown in Fig. 3(e). These results indicate that the druse is more reflective than the normal disc substance in the 530 nm to 580 nm range. Note that ordinary macular soft drusen studied with a previous CTIS hyperspectral system also showed a reflectance peak around 560 nm [17], consistent with a similar biochemical composition of the two types of drusen.

A time-lapsed video of oxygen saturation dynamics was then recorded with a frame rate of 5.2 fps. The oxygen saturation map (see Fig. 4 ) was calculated at each scene with the algorithm detailed in Refs. [1,18]. The oxygen saturation dynamics at one arteriole (pointer, Fig. 4) is shown in Fig. 5 . These results show that the relative saturation index value reaches its maximum at time t = 720 ms and t = 2340 ms, an interval of 1.6 sec. Interestingly, the subject’s pulse of 72 bpm would suggest that the O2 saturation should peak about every 0.84 sec, or about twice as often as measured herein. A possible explanation may lie in the anomalous circulation on the optic disc in subjects with optic disc drusen [14,19], and in fact the central retinal artery can be bypassed completely [19], but this of course is speculation. Note that during the acquisition period, the eye was constantly moving (Media 2). Since current scanning-based hyperspectral retinal cameras normally require an exposure time > 5 secs long to acquire an (x, y, λ) datacube [20], all in vivo retinal data from such systems would be compromised by motion artifacts. In particular, a dynamic study such as seen in Figs. 4 and 5 would not even be possible.

Fig. 4

Oxygen saturation dynamics near the optic nerve (Media 2)

Fig. 5

Relative saturation index vs. time at an arteriole on the optic nerve

Next we measured the absorption spectra of macular pigment (MP) in the central macula (see Fig. 6 ). Previous studies have shown that accurately mapping MP optical density and characterizing its biochemical nature would be instrumental in understanding the role of MP in age-related macular degeneration [5,21]. Using the absorption spectrum of MP is widely considered as a clinically applicable and accessible tool in MP quantification [21]; however, few experiments can fully implement this measurement in vivo—they are either restricted in spatial extent to single line scans [22] or spectrally to a limited number of spectral bands [23]. Here we demonstrated that the proposed system is capable of measuring the absorption spectrum of MP non-invasively. In the experiment, a field-of-view in the central macula (Fig. 6 (a)) was acquired by the hyperspectral retinal camera with a single snapshot. The spectral range 475 nm–520 nm was chosen for MP’s absorption spectrum analysis because previous studies indicated that MP has an absorption peak in this spectral range and is almost zero beyond 520 nm [5,21]. The absorption spectrum of macular pigment was calculated by subtracting the measured normalized reflectance spectrum from the normalized lamp’s illumination spectrum in the range 475 nm–520 nm and averaging over pixels in the circled area. The resulting absorption spectrum, with a peak around 490 nm in Fig. 6 (b) and which also drops to zero beyond 500 nm, is consistent with (although not identical to) previous measurements in vitro [5]. The direct recovery of the MP absorption signature by the proposed system thus provides a simple approach for characterizing MP in vivo. Note that the other MP absorption peak around 460 nm [5] was not detected because it is blocked by a 470 nm long-pass filter inside the IMS and thus located outside the measured spectral range of the current system.

Fig. 6

Measured absorption spectrum of macular pigment. (a) Panchromatic image display of acquired (x, y, λ) datacube. (b) Absorption spectrum at circle area A.

4. Conclusions

In summary, we present an IMS-based hyperspectral retinal camera which can acquire an (x, y, λ) datacube of the retina in a single snapshot. Since no scanning is employed, this hyperspectral retinal camera does not suffer motion artifacts or pixel misregistration problems which are inevitable in scanning-based hyperspectral retinal cameras. The acquisition of multichannel retinal images and the recovery of the spectral signatures of oxy-hemoglobin, macular pigment and optic disc drusen demonstrated the capability of this device in eye imaging applications. To the best of our knowledge, this is the first report of the spectral reflectance of optic disc drusen; combined with previous work on the spectral signature of macular soft drusen in age-related macular degeneration (AMD), it is compatible with biochemically similar components of the two species. Additionally, for the first time we have successfully monitored oxygen saturation dynamics in vivo with sub-second temporal resolution. Such dynamic data would be of crucial value for physicians who need to monitor improvement or setbacks in retinal oxygenation after therapies directed towards retinal vascular diseases. To implement this hyperspectral retinal camera in clinical diagnosis, trials are underway at the Harkness Eye Institute in Columbia University to correlate observed retinal spectral signatures with specific ocular diseases, particularly retinal vascular disease and the drusen of AMD, and establish a significant patient database for future non-invasive ophthalmological diagnosis.