Line-scanning confocal microendoscope for nuclear morphometry imaging.

Fiber-optic endomicroscopy is a minimally invasive method to image cellular morphology in vivo. Using a coherent fiber bundle as an image relay, it allows additional imaging optics to be placed at the distal end of the fiber outside the body. In this research, we use this approach to demonstrate a compact, low-cost line-scanning confocal fluorescence microendoscope that can be constructed for <$5000. Confocal imaging is enabled without the need for mechanical scanning by synchronizing a digital light projector with the rolling shutter of a CMOS camera. Its axial performance is characterized in comparison with a nonscanned high-resolution microendoscope. We validate the optical sectioning capability of the microendoscope by imaging a two-dimensional phantom and ex vivo mouse esophageal and colon tissues. Results show that optical sectioning using this approach improves visualization of nuclear morphometry and suggest that this low-cost line-scanning microendoscope can be used to evaluate various pathological conditions. (2017) COPYRIGHT Society of Photo-Optical Instrumentation Engineers (SPIE).

Introduction

Endomicroscopy in combination with molecular probes has provided clinicians with a powerful tool to visualize tissue architecture and cellular morphology to investigate disease progression. In probe-based endomicroscopy, a coherent fiber bundle is used to enable microscopic imaging with subcellular resolution through the working channel of a standard endoscope. Due to its minimal invasiveness, it is widely applicable in the evaluation and management of many clinical conditions, such as early detection of neoplasia in the gastrointestinal tract, cervix, pancreas, and lung. Existing commercial and research platforms, such as the Cellvizio endomicroscopy system (Mauna Kea Technologies, Paris, France) and the high-resolution microendoscope (HRME), offer an opportunity to provide real-time histological information.

The coherent fiber bundle used in probe-based endomicroscopy serves as an optical image relay that allows for external implementation of sophisticated opto-mechanical systems at its proximal end. Optical sectioning, for example, can be introduced to increase the axial resolution via structured illumination or confocal scanning., The resulting benefits, as demonstrated in a range of laboratory and clinical studies, are manifold. When used with topical staining, optical sectioning has been shown to reduce out-of-focus light and improve image contrast; in applications that require the use of IV staining such as fluorescein, optical sectioning is critical for the rejection of high background signals.,, It is also particularly desirable in imaging highly scattering tissues with crowded nuclei, such as in regions of precancer or cancer., The enhanced ability to resolve individual nuclei can also potentially facilitate development of automated algorithms to diagnose diseases based on cell morphology such as nuclear size, density, and nuclear-to-cytoplasmic area ratio., In addition, subsurface tissue imaging can be realized in confocal endomicroscopy by incorporating depth focusing objectives at the distal fiber end.,,

The implementation of optical sectioning, however, usually requires integration of complex optomechanical components to rapidly scan and descan individual fibers in the coherent bundle, which adds to system cost. Alternatively, the rolling shutter of a CMOS detector can be used to achieve versatile slit detection without the need for a physical aperture. On the illumination end, a digital light projector (DLP) can be synchronized as a spatial light modulator to project matching illumination lines and perform confocal imaging., In this work, we present the first demonstration of a line-scanning confocal microendoscope based on a DLP and a CMOS camera without the need for mechanical scanning. We characterized its axial performance in comparison with a nonscanned HRME (or the standard HRME), and we validated its optical sectioning capability by imaging a two-dimensional (2-D) phantom and ex vivo mouse esophageal and colon specimens. The system offers high-resolution endoscopic imaging with optical sectioning and can be built into a compact enclosure at a low cost ().

Methods

Optical Setup

The standard HRME is described in detail elsewhere. Briefly, the HRME probe is a coherent fiber bundle consisting of 30,000 individual fibers with a core-to-core spacing of and a circular field of view (FOV) of (IGN-08/30, Sumitomo Electric Industries). An LED centered at 455 nm (M455L2, Thorlabs, Newton, New Jersey) was used to provide illumination and a scientific CCD camera (Grasshopper 2, FLIR Integrated Imaging Solutions Inc., Richmond, Canada) was used for fluorescence imaging.

In the line-scanning confocal HRME, scanning illumination is provided by a DLP. For confocal detection, a rolling shutter CMOS sensor replaces the CCD camera with a global shutter to offer a versatile electrically controllable detection slit. A schematic of the confocal HRME is shown in Fig. 1. A DLP (LightCrafter 4500, Texas Instrument, Dallas, Texas) was used to program the illumination patterns. The blue LED of the DLP light engine is centered at 448 nm with a 16-nm bandwidth and was used for fluorescence excitation. The projected lines were focused through a collimation condenser (, LA1986, Thorlabs, Newton, New Jersey) and a objective (RMS10X, Thorlabs, Newton, New Jersey) on the proximal fiber end; scanning of the illumination across the fiber was achieved by the sequential projection of illumination line patterns. Fluorescence signal collected by the fiber was imaged with a scientific CMOS sensor (Firefly MV USB 2.0, FLIR Integrated Imaging Solutions Inc., Richmond, Canada) through an imaging lens (, LA1257, Thorlabs, Newton, New Jersey). The optical setup at the proximal end, as shown in the solid box in Fig. 1, was housed in a enclosure.

Fig. 1

Schematic of the line scanning confocal HRME platform. Solid arrows show directions of scanning on the DLP, CMOS, and fiber surface; dashed arrows show the data flow between the DLP, camera, and laptop. The solid box indicates the optical system enclosure.

A laptop was used to program the DLP pattern sequence and retrieve the camera images. During image acquisition, a transistor–transistor logic signal was sent from the DLP to trigger the camera exposure; the rolling shutter was synchronized with the DLP sequence to perform confocal imaging.

Synchronization and Confocal Imaging

The CMOS sensor offers a rolling shutter that can be used as a versatile slit for confocal detection. The exposure of rows on a CMOS sensor is activated sequentially at a fixed line frequency (for the Firefly MV USB 2.0, 16231 Hz), as illustrated by the green parallelogram in Fig. 2(a). As exposure begins on each row, photons are acquired for a predefined shutter time before the readout takes place. With a fixed line frequency, the number of rows under exposure concurrently (), and hence the width of the rolling shutter aperture, can be controlled by adjusting the shutter time : To project line-scanning illumination, the DLP is programmed to perform in pattern sequence mode. Each illumination sequence is saved as a one-bit pattern that can be projected by toggling on and off an array of micromirrors (for the DLPLightCrafter 4500, ). The LightCrafter 4500 frame buffer supports storage of two 24-bit images, allowing up to 48 continuous illumination sequences during a single frame acquisition. The size and optical magnification of projected illumination lines were adjusted so that they match the aperture width .

Fig. 2

Synchronization of the CMOS and DLP and construction of a confocal image by averaging two complementary images. Imaging configuration I in (a) shows the spatiotemporal coordination of the DLP illumination lines with the CMOS rolling shutter aperture; the discrete nature of DLP scanning combined with the continuous sequential readout of CMOS rows introduces nonuniform exposure among the rows, as revealed in configuration I in (c) and the corresponding image (d). A complementary image (e) can be acquired by shifting the exposure distribution by in configuration II in (b). A confocal image without illumination artifacts is then constructed by averaging two complementary frames, as shown in (d), (e), and (f). Lens paper was imaged as a target. , number of rows in the detection aperture; , CMOS exposure time.

The spatiotemporal coordination between illumination and detection is illustrated in imaging configuration I in Fig. 2(a). As shown in the green parallelogram, the rolling shutter aperture contains rows undergoing exposure simultaneously and scans across the active FOV (for the Firefly MV USB 2.0,960 rows) row by row. To cover a total number of 960 rows using 48 DLP sequences in a nonoverlapping manner, the illumination lines with a matching size (, , , etc.) are scanned with a step no fewer than 20 rows, each of them centered at rows , , , etc. Meanwhile, the projection of each illumination line is synchronized with the exposure of the pixel row at its center. Illumination line , for example, is only projected when its central row (row ) undergoes exposure. The remaining illumination lines are projected in a similar manner without overlap.

The discrete nature of DLP scanning combined with the continuous sequential readout of CMOS rows introduces nonuniform exposure among the rows. The effective exposure time , defined as the temporal overlap between the CMOS exposure and DLP illumination [the green parallelogram and blue rectangles in Fig. 2(a), respectively], varies for each row along the scanning direction. Specifically, is maximal at the illumination central rows [i.e., rows , , , etc. in Fig. 2(a)] and drops by half at the borders of the illumination lines as shown in Fig. 2(c). The resulting image reveals illumination artifacts in Fig. 2(d).

A complementary image can be used to eliminate the illumination artifacts. To capture an image with a complementary distribution in Fig. 2(c), the illumination lines are physically shifted by rows while maintaining the temporal synchronization in imaging configuration II as shown in Fig. 2(b). By averaging the two frames in Figs. 2(d) and 2(e), a confocal image without illumination artifacts is constructed in Fig. 2(f). Both imaging configurations were programmed in the DLP, so the complementary images can be obtained with two consecutive frames. The averaged images were displayed in a MATLAB Graphical User Interface (The MathWorks, Natick, Massachusetts).

Characterization of Axial Sectioning Performance

The optical sectioning capability of the confocal HRME was evaluated and compared with a standard HRME. The axial resolution was also measured as the optical slit size was varied by tuning the shutter time and adjusting the DLP sequences.

To accurately evaluate the optical sectioning performance, a mirror (PF10-03-G01, Thorlabs, Newton, New Jersey) was used as a target. The system was converted into reflection mode to image the mirror by removing the emission filter. The mirror was mounted to a stepper motor (LSA10A-T4, Zaber Technologies, Vancouver, BC, Canada) and was initially placed in gentle contact with the fiber bundle surface. It was then moved away from the fiber in increments up to . The image intensities were measured at each axial distance. The background signal, mainly from internal reflections, was determined with the mirror removed and subtracted from all measurements. This was repeated for slit widths of 20, 40, 60, 80, and 100 pixel rows, with the 20-pixel slit as the smallest aperture size to scan the entire FOV.

Two-Dimensional Phantom Validation and Ex Vivo Imaging

The optical sectioning performance in fluorescence mode was first evaluated by imaging a 2-D phantom. The fluorescence phantom was developed as previously described. Briefly, of fluorescent polystyrene microspheres (F-21010, Thermo Fisher Scientific, Waltham, Massachusetts) in solution were pipetted onto a glass slide, which was then allowed to dry for about 20 min prior to imaging. Similar to the mirror, the 2-D phantom was first imaged in gentle contact with the fiber and then imaged each time the fiber was retreated by . This was repeated for slit widths of 20-, 40-, and 60-pixel rows.

The confocal HRME was also evaluated by imaging excised mouse tissue ex vivo. Mice were acquired from Jackson Laboratories. Squamous epithelium in the esophagus and columnar epithelium in the colon were imaged after topically staining with proflavine (0.01% w/v in phosphate buffered saline). The contrast in images acquired with the standard and confocal HRME was compared. In squamous epithelium, the nuclear and cytoplasmic regions were manually segmented and the nuclear-to-cytoplasmic signal ratio was calculated. In columnar epithelium, the contrast between the glandular walls and lumens in line scans was quantified and compared. All animal experiments were reviewed and approved by the Institutional Animal Care and Use Committee of Rice University.

Results

Axial Sectioning Performance

The sectioning profiles with and without the confocal line-scanning are shown in Fig. 3. Without slit detection, the standard HRME demonstrated minimal out-of-focus signal rejection. When the line-scanning was introduced, the background rejection was significantly improved in the confocal HRME. The optical sectioning was most prominent using a 20-pixel slit on the CMOS sensor, which corresponded to about on the fiber surface; as the confocal aperture size was increased, the axial performance transited from the confocal to the nonconfocal regime.

Fig. 3

Optical sectioning profiles of the standard HRME (no slit) and confocal HRME (20, 40, 60, 80, and 100-pixel slit). The line scanning confocal configurations significantly improve the axial resolution of the standard HRME. As the slit size increases, the axial performance transits from the confocal to nonconfocal regime. A 20-pixel slit on the CMOS sensor is about on the fiber surface.

Two-Dimensional Phantom and Ex Vivo Validation

The findings above were validated by imaging a monolayer of beads in fluorescence mode. In each column of Fig. 4, the image at was normalized to itself, and images at greater axial distances were adjusted equally. Compared with the image at , the corresponding image at 80 and appeared defocused and revealed intensity loss. The signal loss was minimal using the standard HRME (no slit) and was most striking when a 20-pixel slit was used. These findings were consistent with the axial profiles in Fig. 3 and confirmed that a small detection slit improves optical sectioning capability.

Fig. 4

2-D phantom images with varied slit widths. Images acquired at a defocus of 0, 80, and are shown for the standard HRME and two confocal configurations (60-pixel and 20-pixel slits). Compared with the image at , the corresponding image at 80 and revealed signal loss and indicated rejection of background signal. The signal loss was most significant using the 20-pixel slit, suggesting that a smaller detection slit in the confocal configuration improves optical sectioning capability.

Based on the results above, a 20-pixel slit was used in subsequent evaluation of the imaging performance in highly scattering tissue samples. Figure 5 shows images of mouse squamous esophageal epithelium acquired by the standard (A) and confocal (B) HRME. Both images were normalized for visual comparison. Compared with the standard HRME, the confocal HRME demonstrated improved rejection of out-of-focus signal and enhanced image contrast. The enhancement was most striking in regions indicated by the white arrows, with the confocal image showing clearly defined nuclei that were difficult to resolve in the standard HRME. The difference was quantified by manually segmenting the nuclear and the cytoplasmic regions; the average intensity of nuclear and cytoplasmic regions was normalized and shown in Fig. 5(c). The ratio between the nuclear-to-cytoplasmic signal in the standard and confocal HRME was 1.19 and 1.51, respectively.

Fig. 5

Ex vivo images of mouse squamous esophageal epithelium using (a) the standard and (b) confocal HRME. The confocal HRME resolved nuclei with enhanced contrast, especially in regions indicated by the arrows. (c) The normalized intensities of nuclear and cytoplasmic regions are shown; the error bars show the intensity standard deviation in these regions. The resulting ratio of the nuclear-to-cytoplasmic signal in the standard and confocal HRME was 1.19 and 1.51, respectively.

Images of ex vivo mouse columnar colon epithelium are shown in Fig. 6. The confocal image revealed significant background rejection, which was most prominent in the lumens. In addition, individual nuclei on the glandular walls were more readily resolved. To evaluate this difference, intensity profiles were plotted for two line scans as shown in Figs. 6(c) and 6(d) (corresponding to white lines on the left and right, respectively). Regions for the glandular walls and lumens were identified and indicated by the brackets; the contrast was quantified by comparing the average intensity of the glandular wall (maxima) to the lumen (minima). The ratio between the two was calculated for 12 pairs of maxima and minima in two line scans. The resulting gland-to-lumen ratio was approximately twice as high in the confocal as in the standard HRME ( in standard and in confocal; ).

Fig. 6

Ex vivo images of mouse columnar epithelium using (a) the standard and (b) confocal HRME. Profiles are shown for the line scans on the (c) left and (d) right. The brackets indicate the glandular walls and lumens. The resulting gland-to-lumen ratio was higher in the confocal than the standard HRME ( in standard and in confocal; ).

Discussion and Conclusion

In this study, we report the development and ex vivo validation of a low-cost confocal microendoscope that integrates a digital light modulator and a CMOS sensor in a compact design. The optical sectioning capability of the confocal HRME improved the visualization of cell architecture, especially in crowded regions. In addition, quantitative analysis reveals an enhancement in parameters, such as the nuclear-to-cytoplasmic ratio, which can potentially facilitate automated objective diagnosis based on cell density and morphology.,

The current setup reduces the complexity and cost of confocal imaging by eliminating the need for mechanical scanning. A tradeoff is a reduced frame rate limited by the CMOS and DLP scanning speed. In trigger mode, the CMOS sensor used in this research works at a frame rate of 7 to 8 fps; since two consecutive frames need to be averaged, the frame rate is further cut down to 3 to 4 fps. While this is not problematic for imaging ex vivo specimens as seen here, motion artifacts will likely impair image quality in vivo. For these applications, the imaging speed can be accelerated with a faster CMOS sensor and DLP. The frame rate can also be potentially doubled if the illumination artifacts on a single frame can be erased by adjusting the local gain based on the exposure distribution.

While ex vivo imaging confirmed the benefits of optical sectioning, the axial resolution of the confocal HRME can be further improved to better reject background signal. Measured axial profiles suggest that the system sectioning performance could be increased by reducing the detection slit size, while it is ultimately limited by the core spacing of the fiber bundle. This can be realized in the future with a faster DLP supporting sequential projection of added patterns, although it may increase the system cost. Alternatively, if there is only minimal sample motion, the out-of-focus signal can be further reduced at the expense of a lower frame rate. For example, the residual background signal during line-scanning can be recorded with a detection slit offset, and an image subtraction offers an axial resolution comparable to a point-scanning confocal microscope., In a different approach, multifocal scanning can be introduced via the DLP and consecutive images can be multiplexed to reconstruct a confocal image with improved background rejection.,

In conclusion, the work here presents an affordable and portable microendoscope capable of confocal imaging. The versatility of a digital illumination modulator and an electrical detection aperture permits convenient implementation and evaluation of varied optical arrangements. Further optimization is required to increase the current frame rate for in vivo applications. Future work will also be needed to explore the potential benefits of optical sectioning in a variety of pathological conditions.