Structured oblique illumination microscopy for enhanced resolution imaging of non-fluorescent, coherently scattering samples.

Many biological structures of interest are beyond the diffraction limit of conventional microscopes and their visualization requires application of super-resolution techniques. Such techniques have found remarkable success in surpassing the diffraction limit to achieve sub-diffraction limited resolution; however, they are predominantly limited to fluorescent samples. Here, we introduce a non-fluorescent analogue to structured illumination microscopy, termed structured oblique illumination microscopy (SOIM), where we use simultaneous oblique illuminations of the sample to multiplex high spatial-frequency content into the frequency support of the system. We introduce a theoretical framework describing how to demodulate this multiplexed information to reconstruct an image with a spatial-frequency support exceeding that of the system's classical diffraction limit. This approach allows enhanced-resolution imaging of non-fluorescent samples. Experimental confirmation of the approach is obtained in a reflection test target with moderate numerical aperture.

1. Introduction

Microscopy is critical in the biological sciences for its ability to visualize biological samples at the cellular level. There are many subdivisions under this umbrella of general microscopy, and each are tailored towards specific visualization, design, and contrast requirements. Examples that have found widespread use include dark-field, phase-contrast, holographic, and fluorescent microscopies [1]. However, a critical factor that physically limits the optical resolution of microscopy in general is diffraction [2]. Unfortunately, many biologically relevant structures are at sizes beyond the spatial frequency support of this diffraction limit and are thus physically unobservable using conventional optical techniques. This has prompted many attempts to surpass this limit to achieve super-resolution.

These efforts have found tremendous success in fluorescent imaging, where properties of fluorophores are creatively utilized to visualize the sample beyond the diffraction limit. One broad category of such super-resolution techniques relies on single molecule detection, and uses techniques, such as photoswitching, to excite individual molecules at different times. Individual emitters can thus be localized with subdiffraction resolution and composite images incorporating information from the entire acquisition can be reconstructed. Examples of such techniques include photoactivated localization microscopy (PALM) and stochastic optical reconstruction microscopy (STORM) [3-6]. Another main category of super-resolution techniques uses patterned excitation to spatially modulate the excitation of the sample. Two such techniques that have garnered much attention, stimulated emission depletion (STED) and ground-state depletion (GSD) microscopy, modulate between the transition states of the fluorophores to optically narrow the excitation point-spread-function (PSF) [7,8].

A particularly robust and efficient solution for far-field super-resolution that also relies on patterned excitation is structured illumination microscopy (SIM). Conventional SIM typically illuminates the sample with a sinusoidal pattern. In the linear case, the resulting emission is a product of the sample structure and the excitation pattern [1-10]. In frequency space, this emission pattern contains sample frequency components that have been aliased into the system’s passband from beyond the diffraction limit. Acquiring multiple images with shifted illumination patterns allows demodulation of the high frequency content from the diffraction-limited content, and after appropriate shifting in frequency space, an image with subdiffraction resolution can be reconstructed [11]. Rotating the sinusoidal illumination pattern allows isotropic filling of frequency space, and the reconstructed image is thus super-resolved in all orientations. With linear structured illumination, the maximum super-resolution gain is approximately a factor of two. Nonlinearities in the excitation/emission of particular fluorophores can be exploited to achieve even greater gains, and super-resolution by factors of four or five have been reported using nonlinear processes such as saturation and photoswitching [12-14].

The super-resolution techniques introduced above require fluorescence and are thus ill-suited for samples that are highly scattering but are either not autofluorescent or difficult to fluorescently tag. A particularly exciting extension to conventional SIM that we explore here is its application to non-fluorescent, coherently scattering samples. The idea of coherent SIM has been introduced in [15-19]. Here, however, we provide a more extensive theoretical framework, include quantitative experimental findings that support the theory, and demonstrate enhanced resolution of an ex-vivo histological sample. We also draw a parallel between coherent SIM and oblique illumination microscopy. We show that coherent SIM simultaneously illuminates the sample with multiple oblique beams, each of which results in a shifted region of the sample’s frequency spectrum being diffracted into the system aperture and multiplexed, enabling the reconstruction of an enhanced-resolution image. Given the obliquities of the individual illuminations, we note that each resulting region of multiplexed frequency content is accounted for by Abbe’s diffraction theorem. Thus, coherent SIM is not a super-resolution technique in the conventional sense for it does not gain sample information from beyond Abbe’s diffraction limit; however, similar to conventional SIM, it allows reconstruction of an image equivalent to what would be obtained with orthogonal illumination but with an enhanced detection passband.

2. Theory

2.1. Mathematical framework

We first note that a sample under spatially coherent illumination will scatter coherently, such that scatterings from different sample locations will have fixed phase relationships with each other. Therefore, for spatially coherent illumination, the image field is a linear transform of the sample field. This is in contrast with conventional SIM, where the linear relationship is between the image and object intensities [9,11] because of the spatial incoherence of the detected fluorescence. Thus, disregarding magnification and operating in the paraxial regime, the image intensity measured by a detector in a coherent illumination/detection system is given by the nonlinear relationwhere ⊗ is the convolution operator, r is the spatial position vector, is the image intensity distribution, is the object reflectance/transmittance function, and is the illumination field distribution to be imaged onto the object. We assume that the illumination and detection paths of the system share a limiting circular aperture, defining the coherent point-spread-function (PSF) . After Fourier transforming, the image spatial frequency distribution can be written aswhere is the spatial-frequency vector, is the normalized frequency transfer function corresponding to the system PSF and Y(), X(), and I() are the frequency distributions of the detected image intensity, object reflectance/transmittance, and illumination field, respectively.

In the case of orthogonal plane wave illumination, and the resulting image frequency distribution is given by

From coherent diffraction theory, it is known that the transfer function is a real function given by the system’s pupil function [20]. Thus, from Eq. (3), it is clear that, under orthogonal illumination, defines the passband of the system’s diffraction limit by sharply rejecting field frequencies with magnitude beyond some cutoff but allowing all other frequencies to pass through unchanged. We introduce a framework below to use the same physical aperture and still reconstruct an image equivalent to one obtained with orthogonal illumination but an enhanced passband.

For the sake of simplicity, however, we first consider orthogonal illumination through the enhanced passband, defined by the extended transfer function,where and are two mutually orthogonal frequency vectors of equal magnitude such that and . It is clear that this extended transfer has a larger frequency support than and will thus lead to an image more highly resolved than . We write this corresponding “enhanced-resolution” image as and use Eq. (4) to expand,where ⋆ is the correlation operator. By grouping the terms together within the square brackets, Eq. (5) can be directly written as a sum of nine “enhanced-resolution” components, ,We next show that these enhanced-resolution components can be reconstructed using patterned illumination of the object through the original aperture, and thus can be reconstructed without the extended aperture.

We first consider a 2D structured illumination pattern through the original aperture, given byFourier transforming, substituting into Eq. (2), and mathematically expanding, we write the corresponding intensity image ,We define the terms grouped together within the square brackets so that Eq. (8) can be directly written as a linear combination of nine “patterned-excitation” components ,where refer generally to the phase coefficients in Eq. (8). We mathematically show in the Appendix that these component terms are directly related to and can be exactly written in terms of the enhanced-resolution component terms introduced in Eq. (6). Using the relations presented in the Appendix, we rewrite Eq. (8) as a linear combination of the enhanced-resolution components,The enhanced-resolution components can be linearly solved for by taking at least 9 acquisitions with independent phase shifts . Statistical or signal processing techniques, similar to ones introduced for conventional SIM in [21], may be applied at this point to optimize SNR or reduce the required number of raw acquisitions. After appropriately calculating and shifting these enhanced-resolution components in frequency-space and summing, can be reconstructed via Eq. (6).

In Fig. 1 below, we show an extended transfer function and a simulated coherent SIM reconstruction of a 1951 USAF test chart. We note that the amplitude of the extended transfer function shown in Fig. 1(a) below is color-coded in field, not intensity. As was explained above, because of the coherent imaging of the system, the image and object fields are linearly related, and thus the concept of a linear transfer function is only applicable in the field regime. In this regime, the simulated diffraction limit was set to and the illumination frequency set to . The effects of enhanced-resolution can be clearly seen when comparing the modulation of the Group −1 El 4 set of bars in the intensity image simulated in Figs. 1(e)–1(g).

Fig. 1

Numerical simulation showing the extended transfer function and its enhanced-resolution reconstruction ability. (a) The extended transfer function given by Eq. (4) and the intensity of the associated structured illumination field given by Eq. (7). The transfer function’s axes are given in multiples of . The green dashed circle outlines the frequency support of the original diffraction limit. (b,c,d) True, orthogonally illuminated, and enhanced-resolution images, respectively, of a sample USAF test chart. (e,f,g) Magnified view of Group −1 El 4 set of bars at 0.71 lpmm from (b,c,d), respectively. Note the enhanced-resolution capabilities shown in (d,g).

We now emphasize a fundamental concept that directly arises from this framework. We note that the last four enhanced-resolution components in Eq. (6) contain cross-correlation terms involving orthogonal frequency shifts of . These terms require the orthogonal shifts to be simultaneously aliased into the system’s detection passband, and imply that cannot be reconstructed by simply illuminating the sample with a rotating sinusoidal pattern. This is in contrast with conventional SIM, where 2D frequency space is typically filled by illuminating the sample with a rotating sinusoid. This key difference arises due to the coherent imaging of the sample, where the relationship between the object transmittance and detected image intensity is intrinsically nonlinear. Thus, the illumination pattern itself must contain the same frequency components around which enhanced-resolution is required. In the example above, we used two orthogonal frequency components in the illumination to fill frequency space. It naturally follows that, to have a more isotropic filling of frequency space, more frequency components at different rotation angles could be used in the illumination pattern. They would, however, all need to be present simultaneously

2.2. Comparison to oblique illumination microscopy

We now note an interesting observation that allows a more physical understanding of coherent SIM. Generating the structured illumination pattern given in Eq. (7) requires the interference of four separate, coherent beams at the sample plane, but not orthogonal to it. Because scattering, and not fluorescence, is being detected, this setup is in essence oblique illumination microscopy, albeit with multiple illumination beams. For each individual illumination beam, a diffraction limited region of the sample’s spatial-frequency spectrum is transmitted through the optical system; however, each region has a unique position in the sample’s complete spectrum, depending on the obliquity of its corresponding illumination beam, and may include high spatial frequencies not present in other regions [15]. Thus, due to the multiple illumination beam superposition at the sample, each raw image contains multiplexed regions of spatial frequency content that are individually diffraction-limited and have identical frequency support, but are shifted over different regions of the sample spectrum. This is in contrast to the true super-resolution provided by conventional SIM, where the reconstructed high spatial frequencies are unattainable regardless of the obliquity of any single illumination beam. However, though the coherent SIM reconstructed image does not contain any “super-resolved” frequencies, it has an enhanced frequency support with up to twice the extent of the diffraction-limited support of any single-shot image. Therefore, analogous to conventional SIM, a coherent SIM reconstructed image is equivalent to an image obtained with orthogonal plane wave illumination with an enhanced detection passband.

3. System design

To experimentally test our framework for coherent structured illumination enhanced-resolution, we used a pixel-addressable spatial light modulator (Digital Light Processing (DLP), Texas Instruments) to gain custom control over the structured pattern imaged onto the sample. A schematic of our optical system is shown in Fig. 2 . We note here that for the sake of simple theoretical verification, a low numerical aperture lens was used to image the sample. The resulting modest diffraction limit was used to avoid optical aberrations and other experimental non-paraxial imperfections. The spatial light modulator (DLP) was aligned so that the plane of the chip was orthogonal to the optical axis. The DLP was programmed with a periodic 2D grid pattern and was illuminated with a Gaussian single mode, spatially coherent, and collimated 405 nm laser beam, resulting in 2 sets of diffraction orders at orthogonal orientations. The illumination angle onto the DLP chip was aligned such that the diffraction orders were centered around the optical axis. These diffraction orders (blue) were directed through a relay of two 4-f systems, and then recombined to coherently interfere at the sample. A mask was used at the Fourier plane of the first 4-f system to allow through only the +/− 1 orders from both orthogonal diffraction sets to achieve the 2D sinusoidal illumination as given by Eq. (7). The coherent scattering from the sample (green) was imaged by the second 4-f system and a beam-splitter onto the detector (Pixelink, Ottawa, ON). An adjustable iris was used in the Fourier plane of the second 4-f system to act as the common limiting aperture for the illumination and detection arms of the system. We note that due to the diffractive nature of the DLP, the physical positions of the diffraction orders in the Fourier planes are dependent on wavelength. Thus, the mask and iris would require realignment for different illumination wavelengths.

Fig. 2

Schematic structured illumination imaging system with moderate numerical aperture. SM: single mode fiber at 405 nm; CL: collimating lens; DLP: pixel-addressable diffractive element; L1, L2, L3, L4: lens (f = 120, 200, 150, 50 mm); M: mask; BS: beam splitter; LA: limiting aperture; S: coherently scattering sample.

4. Results

4.1. Structured Oblique Illumination Imaging of Calibration Chart

To experimentally verify the theory, we used a 1951 USAF test target as our coherently scattering sample. By adjusting the limiting aperture, we effectively tune the numerical aperture of the last lens before the sample (L4). By adjusting the limiting aperture, we manually set the diffraction limited resolution to be . The illumination frequencies were pushed to the edge of the system’s passband to effectively double the system resolution after coherent SIM reconstruction.

We compared the resolution achieved between the orthogonally-illuminated (BF) and enhanced-resolution (SI) image of the Group 5 Element 4 set of bars of spacing. Figures 3(a) and 3(b) show the BF and SI images, respectively. The dashed lines indicate the regions (averaged over 5 cross-sectional cuts) over which the horizontal and vertical cross-sectional profiles are shown in Figs. 3(c) and 3(d), respectively. The resolution bars, which lie beyond the BF diffraction limit, were clearly resolved after reconstruction. Qualitatively, these experimental results match well with theoretical predictions.

Fig. 3

Experimental data showing (a) an orthogonally-illuminated (BF) image and (b) an enhanced-resolution (SI) reconstruction. (c) Horizontal cross-sectional profiles taken from (a),(b). (d) Vertical cross-sectional profiles taken from (a),(b). (e) Intensity modulations vs bar freq compared between BF and SI.

For a more quantitative analysis, we provide experimental verification for the extended transfer function given by Eq. (4) and shown in Fig. 1(a). To do so, we used the same procedure to resolve other group elements on the test chart and measured their corresponding intensity modulations, defined asHere, are the maxima and minima, respectively, of the imaged test bars. In an incoherent imaging system, where the object and image intensity are linearly related, this intensity modulation measured as a function of the spatial frequency will directly map out the system’s transfer function [22]. In our coherent imaging system, where the object and image field are linearly related and assuming an object with positive and real transmittance, we define an analogous equation to map out a coherent system’s amplitude transfer function,where we make use of the fact that the field amplitude is the square-root of the intensity. It can be mathematically shown that .

Given that the test chart was oriented to contain only horizontal and vertical frequencies (frequencies only along the red dashed line in Fig. 1(a)), we can see from Fig. 1(a) that for passed frequencies. Thus, the detected intensity modulation is theoretically . In Fig. 3(e) below, we plot the experimental intensity modulations observed in the BF and SI images with respect to horizontal bar frequencies. The analogous plot for experimental modulations for vertical bar frequencies is not shown. In both cases, excellent agreement with the theoretical expectations was observed.

4.2. Structured Oblique Illumination Microscopy of Polystyrene Beads and Histological Sample

To demonstrate the utility of this framework on more relevant samples, we image 20 micron polystyrene beads and an ex-vivo histological sample of a mouse bone joint. The histological sample was excised and fixed onto a cover slide. No staining, freezing, or other preparation measures were used. Because the samples themselves had relatively little inherent scattering, the signal was mainly from the specular scattering off the coverslip from around the sample. We expect this issue to not be a problem for biological samples known to be highly scattering. The images from orthogonal-illumination (BF) and coherent SIM imaging (SI) are compared in Fig. 4 below.

Fig. 4

Top: (a) Experimental orthogonally-illuminated (BF) image and (b) enhanced-resolution (SI) reconstruction of 20 µm polystyrene beads. (c) and (d) represent enlarged regions of (a) and (b). (e) shows a comparison of the cross-sectional intensity profiles between the BF and SI images at the locations marked in yellow in images (c) and (d) Bottom: Same as above, but for a histological sample of a mouse joint.

Looking at the close-up of the region in the samples with higher structural intricacy, it is clear from comparing Figs. 4(c) and 4(d) and Fig. 4(c′) and 4(d′) that the enhanced-resolution images (SI) contains significantly more highly resolved structural information than the orthogonally-illuminated images (BF). Figures 4(e) and 4(e′) compare the BF and SI cross-sectional intensity profiles from the locations marked in yellow in Figs. 4(c), 4(d), 4(c′), and 4(d′). As shown, the enhanced-resolution images shows clear intensity modulations, in contrast with the orthogonally-illuminated images, for structures spaced 23.5 µm and 19.1 µm apart (Fig. 4(e) and 4(e′), respectively), which is beyond the diffraction limit set by orthogonal plane wave illumination through the limiting aperture.

4. Discussion

We have shown experimental results that validate the theoretical framework proposed to obtain enhanced-resolution through structured oblique illumination microscopy (SOIM) of coherently scattering samples. We applied this framework to ex-vivo samples and achieved promising enhanced-resolution imaging of the sample. It is worthwhile now to compare some of the implications of this framework with that of conventional SIM theory. In our treatment here, the illumination and detection arms of the system share the same limiting aperture. Thus, the illumination frequencies are limited by the detection bandwidth. Maximal enhanced-resolution gain is therefore reached when the illumination frequencies are pushed to the edge of the detection passband, resulting in an effective doubling of the orthogonally-illuminated system’s passband. This is analogous to conventional linear fluorescent SIM.

A key difference, however, arises when considering the non-linear nature of this coherent illumination enhanced-resolution reconstruction. This nonlinearity manifests in four of the enhanced-resolution components in Eq. (6) containing cross-correlation terms between orthogonal frequency shifts of . Reconstructing these components with patterned illumination through the normal aperture requires orthogonal frequency shifts in the illumination to be simultaneously interfered at the sample and aliased into the system’s detectable bandwidth. This can only be achieved if the illumination pattern itself has the frequency components around which enhanced-resolution is desired. Attempting to achieve enhanced-resolution by filling out 2D frequency space with a rotating 1D sinusoidal illumination, as is typical of conventional SIM, will not allow reconstruction of these cross-correlation enhanced-resolution components, and will thus yield reconstructed images with artifacts. By extension, to isotropically fill out frequency space with more than two orthogonally oriented sinusoids as we have presented above, more orientations can be implemented into the illumination at the cost of a more mathematically complex reconstruction procedure. The number of orientations that can be simultaneously embedded into the illumination pattern, however, is ultimately limited by the resolution of the spatial light modulator.

5. Conclusion

We have described a theoretical framework that extends the concept for obtaining far-field enhanced resolution via structured illumination to include non-fluorescent, coherently scattering samples. We applied this framework to enhance resolution of varying spatial frequency patterns on a grid test target and have demonstrated excellent quantitative agreement between the observed and theoretical modulations of the spatial patterns. We also used this framework to image polystyrene beads and an ex-vivo histological sample and were able to visualize the samples with enhanced resolution. These successful implementations of coherent SIM show potential to apply this framework to enhance resolution of more biologically relevant samples in the future. Particularly because proper fluorescent tagging may be difficult or impossible for many biologically relevant scattering samples, enhanced resolution from scattering may be especially useful.