Nonlinear structured-illumination enhanced temporal focusing multiphoton excitation microscopy with a digital micromirror device.

In this study, the light diffraction of temporal focusing multiphoton excitation microscopy (TFMPEM) and the excitation patterning of nonlinear structured-illumination microscopy (NSIM) can be simultaneously and accurately implemented via a single high-resolution digital micromirror device. The lateral and axial spatial resolutions of the TFMPEM are remarkably improved through the second-order NSIM and projected structured light, respectively. The experimental results demonstrate that the lateral and axial resolutions are enhanced from 397 nm to 168 nm (2.4-fold) and from 2.33 μm to 1.22 μm (1.9-fold), respectively, in full width at the half maximum. Furthermore, a three-dimensionally rendered image of a cytoskeleton cell featuring ~25 nm microtubules is improved, with other microtubules at a distance near the lateral resolution of 168 nm also able to be distinguished.

1. Introduction

The multiphoton excitation (MPE) technique is widely applied in the biological imaging and microfabrication fields. With its superior axial sectioning capability and long excitation wavelength, MPE offers lower photobleaching and minimum invasiveness, and is therefore particularly suitable for imaging thick tissues and living animals. Further, with the two-photon absorption (TPA) confined to the focal volume, MPE provides an ideal solution for the fabrication of high-precision microstructures. However, one shortcoming of conventional MPE scanning is its low throughput due to point-by-point processing [1,2]. Recent studies have shown that temporal focusing MPE microscopy (TFMPEM) can generate widefield and axially-resolved excitation on a plane-by-plane basis [3-12]. Different diffraction components such as a grating [3-9], an optical diffuser [10], and a combination of two prisms and a grating [11] can be utilized to diffract illuminating light frequencies for temporal focusing. The most common temporal focusing configuration uses a diffraction grating to separate frequencies into monochromatic waves at different spatial angles; then recombines them at the front focal plane of an objective lens. As such, only in the focal plane do the different frequencies overlap in phase and produce a short, high-peak power pulse, allowing MPE to simultaneously occur over the entire projection area.

Currently, the lateral and axial spatial resolutions of TFMPEM are restricted by the diffraction limitation and the confined excitation volume to within a few hundred nanometers and few microns, respectively. To improve the axial resolution near the diffraction limitation, using patterned illumination such as structured light and HiLo [13-16] to eliminate the background noise and achieve optical sectioning is a reliable approach. Further, HiLo has also been employed to improve the axial resolution of TFMPEM [17]. However, there are many approaches [18], to overcome the diffraction limitation and achieve superior lateral resolution, including photoactivation localization microscopy (PALM)/stochastic optical reconstruction microscopy (STORM) [5,19-21], super-resolution optical fluctuation imaging (SOFI) [22,23], and (nonlinear) structured-illumination microscopy (SIM/NSIM) [24-28]. PALM and STORM activate sequentially and time-resolve the localization of photon switchable fluorophores to achieve a super-resolution image. However, they require anywhere from a thousand to tens of thousands of images to precisely determine a single molecule’s position. By contrast, SOFI is a technique that computes the temporal autocorrelation of the CCD pixels to determine the exact position of a single molecule. Lastly, SIM uses structured-illumination patterning, such as a sinusoidal pattern, to perform intensity modulation. With this technique, high spatial frequency information that is higher than the spatial cutoff frequency of the microscope is filtered out and modulated to a low frequency region detectable by the microscope. After remapping the high frequency information to its actual position on the frequency domain, a super-resolution image can be reconstructed. Theoretically, the lateral resolution can be enhanced 2-fold in SIM and at most m + 1-fold in NSIM, depending on the nonlinear mth order.

In this study, the lateral and axial spatial resolutions of TFMPEM are simultaneously improved through second-order NSIM and projected structured light, respectively. There are several methods to generate structured illumination, such as the conventional interference method [24-28]; however, this method for SIM requires a complex system setup that precisely shifts the phase delay of one of the interference beams and rotates the illumination pattern for multi-direction lateral resolution enhancement. Further, the initial phase of the illumination pattern must be found prior to the image reconstruction process. In digital light processing techniques, the use of a spatial light modulator (SLM) and digital micromirror device (DMD) have been proposed to generate arbitrary patterns in tandem with initial phase control, different phase shift, and rotation angle of illumination patterns for fast optical switching [29], pattern excitation in optogenetics [6], and high-throughput microfabrication [30]. Consequently, the second-order NSIM with projected structured light can be delivered via a DMD; and further, the conventional diffraction grating can be replaced by a DMD in TFMPEM [31]. Herein, the light diffraction of the TFMPEM and the excitation patterning of the second-order NSIM with projected structured light are simultaneously and accurately implemented via a single high-resolution DMD. Based on the concise TFMPEM configuration, the experimental results reveal that the lateral and axial resolutions are enhanced from 397 nm to 168 nm and from 2.33 μm to 1.22 μm, respectively, in full width at the half maximum (FWHM). Also, improvements to the three-dimensionally (3D) rendered image of a cytoskeleton cell with about 25 nm microtubules demonstrates the efficacy of the proposed TFMTEM setup.

2. Optical setup and principle

2.1. Temporal focusing-based multiphoton excitation microscopy via digital micromirror device

In our previous work [31], an enhanced TFMPEM was assembled in which a high-resolution DMD replaced the blazed grating as the diffraction component to separate the different frequencies, thereby allowing the frequencies to then re-overlap completely at the front focal plane of the objective lens to achieve the narrowest pulse width possible on the temporal focusing plane. The 4.0 μm axial excitation volume of the TFMPEM is comparable to that of a setup incorporating a 600 lines/mm grating; hence, the optical sectioning ability of the proposed setup is validated. Since the DMD is placed on the image-conjugate plane of the objective lens’ focal plane, the arbitrary MPE pattern can be precisely projected on the focal plane. Figure 1 shows the overall configuration of the TFMPEM with the incorporated computer controlled high-resolution DMD. Key components include a Ti:sapphire ultrafast amplifier (Spitfire Pro, Newport, USA) coupled with a Ti:sapphire ultrafast oscillator (Tsunami, Spectra-Physics, USA) as the seed beam of the amplifier, an upright optical microscope (Axio imager 2, Carl Zeiss, Germany), a triple-axis sample positioning stage (H101A ProScanTM, Prior, UK), a piezo stage (NanoScanZ-200, Prior, UK), an Andor EMCCD camera (iXon Ultra 897 EMCCD, Andor, UK) with a camera adapter (T2-T2 SLR 2.5X, Carl Zeiss, Germany) to magnify the images 2.5x, a data acquisition (DAQ) card with a field-programmable gate array (FPGA) module (PCI-7831R, National Instruments, USA), and a high-resolution DMD (DPL7000, Texas Instrument, USA). The amplifier has a peak power of 400 μJ/pulse, a pulse width of 100 fs, and a repetition rate of 10 kHz, which is sufficient for two-photon excitation (TPE) fluorescence with an area larger than 200 × 200 μm. The center wavelength can be adjusted from 750 to 850 nm and the output beam diameter is about 12 mm.

Fig. 1

Schematic of the TFMPEM with incorporated DMD. The red dashed lines represent the mutual image-conjugate planes between the DMD surface and the objective lens’ focal plane.

As can be seen in the above figure, a half-wave plate (HWP) and a linear polarizer (LP) first adjust power and maintain the horizontal polarization of the beam from the amplifier while a mechanical shutter (VS14S-2-ZM-0-R3, Uniblitz, USA) controls the excitation time to prevent unnecessary exposure. The beam, with an initial incident angle, becomes oblique incident on the DMD, where the micromirrors on the chip have a fixed tilt angle of 12°, either toward the light source (ON) or away from it (OFF). The tilt angle is equivalent to the blazed angle of a blazed grating. Herein, the DMD works as a phase grating for temporal focusing and simultaneously provides an amplitude transmittance for structured illumination. The incident angle is adjusted to 22.8° as the diffraction order of 10, the central wavelength of 750 nm of the ultrafast amplifier, and the effective 19.35 μm pitch of the micromirrors. Also, the gray level of the DMD acts like an optical shutter and can be adopted to adjust the illumination TPE intensity; moreover, the 0~255 gray levels represent the different “ON” states for the 8-bit DMD. Further details of the DMD control can be found in our previous works [31,32]. A set of two relay lenses with respective focal lengths of 200 mm and 75 mm were inserted to lengthen the beam path and adjust the beam size to fit the back aperture of the objective lens. The dispersed beam propagates through the 4f setup, which comprises a collimating lens and an objective lens (UPlanSApo 60X/ numerical aperture (NA) 1.2 W, Olympus, Japan). Due to this setup, the DMD was positioned on the image-conjugate plane of the objective lens’ focal plane, as represented in Fig. 1 by the two red dashed lines. By filtering the collected signal through a dichroic mirror and short-pass filter, only MPE fluorescence signals can pass through the objective and imaging lenses into the EMCCD camera. Sequential sectioning images at different sample depths can be obtained by controlling the nano-scale piezo stage in the z-axis via the FPGA, and then rendered into a 3D image. Refractive elements, including the objective and collimating lenses, inherently induce additional group velocity dispersion into the overall optical system. To compensate, a built-in prism pair in the amplifier can be adjusted to approach the optimal pulse width (< 120 fs) according to a developed Michelson-interferometry-based autocorrelator to measure the pulse width of the amplifier on the sample surface [32]; as a result, the strongest signal appears only at the temporal focusing section.

2.2. Second-order nonlinear structured-illumination microscopy

The equations from a linear SIM [25] can be easily extended into a higher order nonlinear expression. For nonlinear TPE, the detected fluorescence signal can be expressed as:where is the local concentration of fluorophores, is excitation intensity, and is the point spread function (PSF) of the fluorescence detection system. With a sinusoidal illumination pattern, generated by the DMD, the TPE intensity is:where is the excitation spatial frequency of the sinusoidal pattern and is the pattern phase. By substituting Eq. (2) into Eq. (1) and converting it with a two-dimensional (2D) Fourier transform, the second-order representation can be given as:where is a constant and is the nonlinear two-photon excitation spatial frequency. From Eq. (3), five different phase shifts of are needed to extract the five frequency components, namely , , , , and . Herein, five phase shifts of 0, 2π/5, 4π/5, 6π/5, and 8π/5 are performed. After extracting the five components, each frequency component was remapped into its real position with appropriate weighting. To precisely remap the five frequency components in the spatial frequency domain, the image size was doubled by adding zeroes before taking the Fourier transform of the structured illumination images. With remapping and weighting all the frequency components, the data were converted to the space domain and an image with a radius of can be reconstructed in the frequency domain. Consequently, a superior resolution image with a lateral resolution enhancement of can be obtained. Based on the theoretical analysis, when equals , which is the cutoff spatial frequency of the microscope, a maximum 3-fold enhancement of the lateral resolution can be achieved. However, the excitation wavelength in TPE is longer than the emission wavelength, i.e. is never equal to . Under the current experimental conditions with the excitation wavelength of 750 nm and the emission wavelength of 570 nm, the maximum projected pattern frequency could approach , i.e. . Nevertheless, the modulation visibility at the higher spatial frequency becomes smaller. Furthermore, the precise structured illumination process only takes place in the focal plane; hence, the detected out-of-focus signals can be considered as background noise. According to [13], the background noise in was rejected. Furthermore, during the extraction process, the five frequency components were calculated by fitting a linear matrix model, enabling the background noise to be diminished via the least-squares fitting processing. At this moment, the axial resolution can be enhanced via the processing.

These processes enhance the lateral resolution in one direction, which in turn depends on the original structured illumination pattern direction. It should be noted that the linear SIM [25] requires different rotations of the structured pattern to enhance the 2D lateral resolution. Herein, four rotations were taken, namely 0°, 45°, 90°, and 135°, with each rotation undergoing five phase shifts. As such, a total of 20 sinusoidal illumination images were required for 2D super-resolution image reconstruction. The frame rate of the TFMPEM with structured illumination via the DMD can achieve 200 frames per second [9], while the frame rate of the super-resolution TPE image based on the second-order NSIM with four rotations is lowered to 10 frames per second.

3. Experimental results and discussions

3.1. Improving the lateral and axial spatial resolutions

In the experiment, the sinusoidal patterns with five phase shifts, 0, 2π/5, 4π/5, 6π/5, and 8π/5, and four rotations, 0°, 45°, 90°, and 135°, are quickly generated via the DMD. To verify the structured TPE illumination, a R6G fluorescence thin film with a thickness less than 200 nm was utilized and excited with the sinusoidal patterns. The illumination patterns were controlled by both the ON-OFF pixel period and the gray level of the DMD. In this experiment, the ON-OFF period covers 16 pixels and sequentially changes the ON-OFF states to perform the phase shifts. 100 μm beads were measured for estimating the PSF of the system, which was then transferred to the spatial frequency domain to become . The current value of is around 2.5 μm −1. To show the two peaks of and in the spatial frequency domain, the value of was adopted as 1.03 μm−1, which is slightly smaller than the 1.25 μm −1 half value of . The of 1.03 μm−1 and the of 2.06 μm−1 are both smaller than the cutoff frequency of 2.5 μm −1; hence, the two peaks can be observed in Fig. 2(b). Figure 2(a) shows the fluorescent intensity of the structured illumination on the R6G thin film. The blue circles are the normalization of the detected intensity on the EMCCD camera, while the dashed line is the fitting by . By converting the Fourier transform of the structured illumination image in Fig. 2(a), a nonlinear second-order peak caused by the intrinsic nonlinear TPE can be easily found, as shown in Fig. 2(b).

Fig. 2

(a) Sinusoidal illumination on the R6B fluorescent thin film. Blue circles: the detected fluorescent intensity; blue line: the fitting by . (b) Fourier transform of the detected structured pattern in Fig. 2(a), in which a nonlinear second-order peak caused by the intrinsic nonlinear TPE is indicated.

To verify the improvement of the lateral and axial spatial resolutions of the TFMPEM via the second-order NSIM, 100 nm fluorescent beads in water, acting as the specimen, were illuminated by the TFMPEM only and then by the TFMPEM with the second-order NSIM. The fluorescent bead solution was dropped and dried on a cover slide, and then sealed in glycerol for refractive index matching. Hereafter, to increase the enhancement of the lateral resolution, was increased to nearly 1.5 μm−1, i.e. ; consequently, the enhancement based on can achieve roughly 2.2. The resulting images from the TFMPEM and the TFMPEM with NSIM are shown in Figs. 3(a) and 3(b), respectively. A region of interest highlighted by the red dashed circles in Figs. 3(a) and 3(b) is zoomed in, as respectively shown in Figs. 3(c) and 3(d). As can be seen in Fig. 3(c), the gathered beads in the TFMPEM image cannot be discriminated because the diffraction limit at the peak emission wavelength of 570 nm and the 1.2 NA is around 300 nm, which limits the lateral resolution of the widefield microscopy. Theoretically, the second-order NSIM can enhance the lateral resolution by a maximum of 2.52-fold with the 750 nm excitation wavelength in our system.

Fig. 3

TPE images of 100 nm fluorescent beads by (a) TFMPEM only and (b) TFMPEM with second-order NSIM. (c) and (d) are the regions within red dashed circles in Figs. 3(a) and 3(b), respectively.

Ten beads were selected from Figs. 3(a) and 3(b) and fitted for a bead lateral profile according to Gaussian profile. Figure 4(a) shows the beads’ profiles and the lateral resolutions of the TFMPEM only and NSIM enhanced TFMPEM, which in FWHM are 397 nm and 168 nm, respectively. Therefore, the lateral resolution is enhanced 2.4-fold, which is in agreement with the theoretical . In the experiment, we attempted to increase to its maximum value. Based on the modulation transfer function theory, the modulation visibility decreases from 1 when the pattern frequency becomes higher; hence, the signal-to-noise ratio and the contrast of the structured TPE pattern on the specimen worsened, and as a consequence, the intensity modulation information was lost due to the induced distortion in the reconstructed image. Furthermore, to avoid photobleaching, the excited power on the sample was controlled to below 10 mW with an exposure time of 10 ms.

Fig. 4

(a) The 100nm fluorescent beads’ measured lateral profiles by TFMPEM alone (blue) and NSIM enhanced TFMPEM (red). The lateral resolution with TFMPEM only and NSIM enhanced TFMPEM are 397 nm and 168 nm, respectively, in FWHM. (b) The R6G thin film’s measured excitation volume axial profile by TFMPEM (black), and the 200 nm fluorescent beads’ measured axial profiles by only TFMPEM (blue) and NSIM enhanced TFMPEM (red). The excitation volume depth is 3.1 μm in FWHM, and the axial resolution with only TFMPEM and NSIM enhanced TFMPEM are 2.33 μm and 1.22 μm, respectively, in FWHM. The circles represent the measured intensities at different lateral and axial positions, while the dashed lines are the fitted curves.

Nevertheless, SIM not only enhances the lateral resolution, but also provides optical sectioning capability [16,25]. The optical sectioning image without lateral resolution enhancement is reconstructed by structured illumination in one-photon widefield microscopy [13]. This requires three phase shift structured illumination images to restore a sectioning image, without considering high frequency components modulated in the low frequency region. The background noise in can be rejected. The patterns can be accurately projected in the focal plane region, ensuring the frequency component between the region is restored by the structured illumination images. Herein, the five frequency components of the second-order NSIM were calculated by fitting a linear matrix model during the extraction process, enabling the background noise to be reduced via the least-squares fitting process. As such, when considering the high frequency component modulated into the region to increase the lateral resolution, i.e. SIM, it can provide not only lateral resolution enhancement but also optical sectioning capability. In this study, the illumination sinusoidal pattern for the second-order NSIM can be clearly projected into the EMCCD in a confined region in depth, and depends on the NA of the objective lens and the excitation fluorescence wavelength [13]. Other fluorescence signals coming from the TPE volume of the TFMPEM, except for the confined region where the pattern can be imaged well, are rejected as background noise. The structured illumination has provided an optical sectioning ability even in widefield one-photon excitation microscopy [13,16,25]. Similar to the structured illumination approach, the TFMPEM with the NSIM can achieve a better axial resolution than conventional TFMPEM alone. To determine the current TPE depth volume of the TFMPEM only, an R6B fluorescent thin film was axially scanned and imaged, after which the TPE fluorescent signal was calculated at each axial position; hence, the fluorescent intensity profile as a function of axial position can be obtained. In this case, the excitation volume in depth based on this profile was 3.1 μm in FWHM, as shown in Fig. 4(b). To verify the axial resolution improvement, a specimen comprising 200 nm fluorescent beads sealed in glycerol was axially scanned and imaged by TFMPEM. Figure 4(b) shows that the axial resolution of the TFMPEM only via imaging the 200 nm fluorescent beads is about 2.33 μm in FWHM, i.e. only the axial information difference from 2.33 μm in depth can be distinguished. By contrast, for TFMPEM with the second-order NSIM, the axial resolution is enhanced to 1.22 μm in FWHM; hence the enhancement factor is 1.9-fold, as shown in Fig. 4(b). The axial resolution enhancement is limited by the excitation wavelength, the objective lens’ NA, and the period of the projected structured light. Similar to the formula in Ref [13], the axial resolution of the TFMPEM with the structured light near the cutoff spatial frequency could be improved to around 700 nanometers. However, in widefield TFMPEM, the diffraction laser profile in the back aperture of the objective lens is a line, and therefore does not cover the whole aperture. As a result, the full NA of the objective lens is not utilized, and so the optimal axial resolution enhancement cannot be achieved at this moment.

3.2. 3D image of the microtubules of cytoskeleton

To demonstrate the superior lateral and axial resolutions of the TFMPEM with the second-order NSIM, the microtubules of a cytoskeleton, labeled with Alexa 488, are imaged in 3D. The diameter of a microtubule is about 25 nm, which is much smaller than the spatial resolution of widefield microscopy. The TPE fluorescence images of the cytoskeleton’s microtubules observed by the TFMPEM and the TFMPEM with the NSIM are shown in Figs. 5(a) and 5(b), respectively. Herein, Figs. 5(a) and the 5(b) have a deconvolution step via the PSF of the system. In Fig. 5(a), the microtubules of the cytoskeleton are too blurry to be distinguished due to the lower lateral resolution and emission light scattering. Axial scanning videos by the TFMPEM and the NSIM enhanced TFMPEM can be accessed via the links (Media 1 and Media 2) in the caption of Fig. 5. As can be seen in Fig. 5(b), the lateral resolution of the TFMPEM via the NSIM is improved, and background signals, such as scattered light and out of focus signals, are also eliminated to achieve superior axial resolution; consequently, the microtubules of the cytoskeleton can be easily observed. The TPE intensity profiles along the intersecting red dashed lines in Figs. 5(a) and 5(b) are shown in Fig. 5(c). Although the microtubules of the cytoskeleton are still much smaller than the spatial resolutions of the TFMPEM with the NSIM, from the red line in Fig. 5(c), some close microtubules with a distance near the lateral resolution of 168 nm can be further distinguished.

Fig. 5

Cytoskeleton TPE fluorescence images by (a) the TFMPEM only (Media 1), and (b) the second-order NSIM enhanced TFMPEM (Media 2). (c) Intensity profiles along the intersecting red dashed line in Figs. 5(a) and 5(b). Blue line: from Fig. 5(a), red line: from Fig. 5(b).

4. Conclusions

The advantages of TPE, such as less photon damage and deeper sample penetration, can be maintained in TFMPEM. In this paper, it was demonstrated that the lateral and axial spatial resolutions of TFMPEM can be improved via the second-order NISM. The light diffraction of the temporal focusing and the sinusoidal nonlinear excitation patterns of the second-order NSIM can be simultaneously implemented via a concise configuration with only a high-resolution DMD. Since the DMD is placed on the image-conjugate plane of the objective lens’ focal plane, a sinusoidal MPE pattern with different rotations and phase shifts can be precisely projected on the focal plane. The lateral resolution is improved from 397 nm to 168 nm via the second-order NSIM, while the axial resolution is sharpened from 2.33 μm to 1.22 μm, for an enhancement factor of 1.9-fold, due the exact projected structured light and the rejection of background signals such as scattered light and out of focus signals. Furthermore, to demonstrate the superior lateral and axial resolutions of the TFMPEM with the second-order NSIM, microtubules of a cytoskeleton cell are observed as axially scanned 3D video. The diameter of the microtubules from the intensity profile is approximately equal to the lateral resolution of 168 nm; nevertheless, some close microtubules with a distance near the lateral resolution can be distinguished.