Fast multicolor 3D imaging using aberration-corrected multifocus microscopy.

Conventional acquisition of three-dimensional (3D) microscopy data requires sequential z scanning and is often too slow to capture biological events. We report an aberration-corrected multifocus microscopy method capable of producing an instant focal stack of nine 2D images. Appended to an epifluorescence microscope, the multifocus system enables high-resolution 3D imaging in multiple colors with single-molecule sensitivity, at speeds limited by the camera readout time of a single image.

Fast and sensitive acquisition of 3D data is one of the main challenges in modern biological microscopy. Most imaging modalities, such as wide-field, laser-scanning confocal, spinning-disk confocal and light-sheet microscopy[1], record information from one focal plane at a time. 3D images are assembled from a focal stack of 2D images acquired using sequential refocusing. This process is time-consuming and can yield ambiguous spatio-temporal information since the focal planes are not recorded simultaneously. Moreover, the fast movement of the microscope stage (or objective) during refocusing can mechanically perturb the sample when using an immersion objective (though this can be avoided by remote focusing[2]). Fluorescent probes can also be excited and photobleached while they are out-of-focus, resulting in wasted photons and information.

To overcome these limitations, multi-focus imaging forms an entire 3D focal stack simultaneously, so that it can be recorded in a single exposure of the camera, letting signal strength and camera frame-rate alone limit acquisition speed. Multi-focus imaging has been applied in fluorescence microscopy using beam splitters to image different focal planes on separate cameras[3]. This method provides excellent light-efficiency and a large lateral field of view, corresponding to the entire surface area of the camera chip. However, refocusing by translating the camera away from the nominal focal plane induces spherical aberration after a refocus of a few microns[2].

Here we report an aberration-corrected multi-focus microscopy (MFM) method, which produces an instant focal stack of high-resolution 2D images simultaneously displayed on a single camera (Fig. 1a–c). It is based on the use of a diffractive grating to form multiple focus-shifted images[4,5], and on aberration-free refocusing[2]. Our system was designed with the following considerations: (i) sensitivity must be optimized to minimize photobleaching and phototoxicity and to enable high-speed imaging of weak fluorescent samples such as single fluorophores; (ii) multiple focal planes must be acquired without aberrations, especially the dominating depth-induced spherical aberration, to avoid loss in resolution and contrast; (iii) the system must be corrected for the chromatic dispersion which arises when a diffractive element is used to image non-monochromatic light.

Fig. 1

Aberration-corrected multi-focus microscopy (MFM). (a) The multi-focus optical elements are appended to a widefield fluorescence microscope after the primary image plane (at the camera port). Two relay lenses (f1=150 and f2=200 mm) create a conjugate pupil plane (Fourier plane) and the final image plane. The multi-focus grating (MFG) is placed in the Fourier plane and followed by the chromatic correction grating (CCG) and prism. A dichroic mirror (purple) splits the color channels onto separate cameras. (b) The MFG splits and focus shifts the sample emission light to form an instant focal series (the multi-focus image) where each focal plane corresponds to a diffractive order of the MFG. Ray colors denote individual focal planes (diffractive orders). The CCG and prism correct the chromatic dispersion, illustrated by rays of wavelengths λmax and λmin, introduced by the MFG. (c) The instant focal stack recorded on the camera is computationally assembled into a 3D volume in a computer. (d) The MFG is a phase-only diffractive grating with etch depth π. The grating function (basic grating pattern) is optimized to distribute light evenly into the central 3 × 3 diffractive orders (m = 0, ±1), which form the nine focal planes. The geometrical distortion of the MFG pattern introduces a phase shift which is dependent on diffractive order, and in the image corresponds to a focus shift Δz × (m + 3 × m). The CCG panels (except the blank, central panel) contain blazed diffractive gratings that reverse the dispersion of the MFG. The prism directs the images to their positions on the camera. (e) Multi-focus image of 200 nm fluorescent beads Δz = 380 nm. (f) Axial (x-z) point spread function (PSF), radially averaged and displayed in log scale. Resolution measured as full width at half maximum of the PSF is 238 nm laterally (x) and 660 nm axially (z). (g) Best focus position is estimated as the maximum of a Gaussian curve fitted to the bead signal when scanned through focus by the stage (Supplementary Video 1). (h) Plot of the best focus position of each plane. The linear curve verifies the constant focus step Δz betweenplanes. Scale bars 1μm.

Our MFM consists of three specially designed optical elements (Fig. 1d) appended to the camera port of a standard, high-resolution epifluorescence microscope. Two relay lenses are also used, to form a secondary pupil plane (Fourier plane) and the final image plane (Fig. 1a). The diffractive multi-focus grating (MFG) is placed in the Fourier plane to form the multi-focus image, and is followed by the chromatic correction grating (CCG) and prism. The MFG performs two distinct functions. Firstly, it splits the fluorescence light emitted from the sample into separate paths, to form an array of N × N images of the sample on the camera. Each image corresponds to a 2D diffractive order (m) of the grating. Here we describe an MFG with nine focal planes, formed by the central 3 × 3 diffractive orders m = 0, ±1. To minimize light-loss we use a phase-only grating of fused silica, with a grating function (Fig. 1d) designed to distribute the sample emission light evenly and efficiently between the nine focal planes. At the design wavelength (515 nm) the measured efficiency of our MFG (custom made by Creative Microsystems) is ~65% with even distribution between images, as illustrated in Supplementary Fig. 1. This is close to the theoretical maximum efficiency (~67%) of this type of grating, but could be improved (theoretically to ~93%) by using a multi-phase element[6]. To implement an MFM with more focal planes one would use a different MFG to image the m = 0, ±1, ±2 diffractive orders, obtaining 5 × 5 = 25 planes, as shown in Supplementary Fig. 1.

The second function of the MFG is to refocus the array of images so that it forms an instant focal series with a constant focus step Δz. When refocusing deep into a thick sample, the microscope objective is used at a focal distance for which it is not designed. This normally gives rise to depth-induced spherical aberration, which deteriorates the image[2]. To avoid this problem we have used the Abbe sine condition[7] to calculate the defocus phase error δφ(z) in the Fourier plane of a point source at defocus z in the sample. We let the MFG apply an equal but opposite phase shift – δφ(z), that entirely reverses the out-of-focus wavefront error (see online methods). Light from the out-of-focus plane thus exits the MFG with a flat wavefront, as illustrated in Supplementary Fig. 2, and is properly focused onto the camera. The phase shift is introduced by a carefully calculated geometrical distortion of the MFG pattern (Fig. 1d), and is dependent on diffractive order so that each order obtains a focus shift Δz × (m + N × m) (Fig. 1c). The magnitude of the MFG distortion determines the step size Δz. To suit a variety of samples and imaging applications, we have designed several gratings, which produce different focus shifts ranging from 250 nm to 2 μm, spanning 2.25 to 18 μm of sample depth per 3D image.

The MFG is in itself sufficient for imaging monochromatic light. However, even across the relatively narrow (~30 nm) wavelength spectrum of a single fluorophore, chromatic dispersion – inherent to any diffractive element[7] – severely deteriorates resolution. To correct the dispersion we have designed the CCG (custom made by Tessera), which is placed after the MFG at a position where the diffractive orders are separated (Fig. 1b). The CCG panels consist of blazed transmission gratings (oriented as shown in Fig. 1d) that reverse the dispersion from the MFG (see online methods). On its own, the CCG would not only remove the chromatic dispersion but unfortunately also reverse the image separation obtained by the MFG. Therefore we have added a multi-faceted refractive prism (custom made by Rocky Mountain Instruments) to direct the diffractive orders to their positions on the camera. The CCG spacing is adjusted to compensate for the additional dispersion of the prism. (The optical properties of the CCG and prism are illustrated in Supplementary Fig. 3.) We hereby obtain an essentially diffraction-limited point-spread function (PSF) for fluorophores across the visible spectrum. For an MFG with Δz= 380 nm and a 100x Nikon oil-objective (NA = 1.4) the full width at half maximum of the PSF is 238 nm laterally and 660 nm axially (Fig. 1e, f and Supplementary Videos 1 and 2). The focus shift between planes is verified in Fig. 1g, h and further discussed in Supplementary Note 1.

To evaluate the performance of our MFM in living samples we imaged S. cerevisiae (Fig. 2) expressing Cse4-GFP, a histone H3 variant that binds to centromeric DNA (Supplementary Note 2). We were able to track the centromeres (each labeled by 30–80 fluorescent molecules[8,9]) in 3D and observe their behavior during cell division[10] with one image (covering a volume of 20 × 20 × 3.5 μm3) recorded every three seconds (exposure time 100 ms). We observed the biphasic behavior of centromeres during cell division (Supplementary Videos 3 and 4), with a fast initial separation speed of 20 nm s−1 over ~2 min followed by a slower separation of 3 nm s−1 over ~15 min.

Fig. 2

Centromere imaging in yeast cells. (a) and (b) Multi-focus transmission and fluorescence images of S. cerevisiae cells expressing Cse4-GFP. (c) Separation of the centromeres during anaphase over time (maximum intensity projections). (d) Movement in 3D of the two centromere clusters (in black and gray). The correlated 3D motion of the clusters can be seen clearly. Lower right: the separation rate of centromere clusters reveals a two-phase process, with a rapid movement (phase I) followed by a slow one (phase II). Insert: average speed during phases I and II (n = 5 cells). Scale bars 5 μm.

High sensitivity and spatio-temporal resolution are paramount in single-molecule imaging, where the acquisition of 3D data is a major challenge[11]. In a single-particle tracking experiment we imaged RNA Polymerase II (labeled using Halo-tag) in the nucleus of a human osteosarcoma cell line[12] (Supplementary Note 3). We were able to image the heterogeneous nuclear mobility of RNA Polymerase II in 3D at 35 frames per second, observing bound and mobile states and transitions between them (Fig. 3 and Supplementary Videos 5 and 6). The depth (~4μm) over which we could detect and reconstruct individual trajectories in MFM compares well to other 3D single-fluorophore localization techniques: ~3 μm for double-helix[13], ~1 μm for astigmatic[14] and ~2 μm for biplane[15] imaging.

Fig. 3

Single-molecule tracking of RNA Polymerase II. (a) Maximum Intensity Projection (MIP) of the 3D volume of the first frame of Supplementary Video 5 on the x-y, x-z and y-z planes. Magenta spots correspond to single Halo-tagged RNA Polymerase II molecules in U2OS cells. The nuclear membrane (green) is visualized using LaminB1-GFP. Scale bar 5 μm. (b) Movement of the single molecule marked by the dashed frame in a, visualized in MIP in the x-z plane. (Same scale as in a). (c) Histogram of diffusion coefficient (n = 109 molecules, from 7 cells). (d) Examples of individual trajectories of RNA Polymerase II. From left to right showing a bound molecule, a diffusing molecule and a molecule with mixed dynamics. Dimensions are in μm. Corresponding 3D temporal sequences are available in Supplementary Video 6.

We also explored MFM of thicker samples with a larger imaging volume of 60 × 60 × 18 μm3, at lower magnification and coarser focal plane separation Δz = 2 μm, to visualize fluorescently labeled neurons in the developing C. elegans embryo (Supplementary Note 4). At nine volumes per second we were able to image the entire late embryo moving rapidly inside the eggshell (Supplementary Videos 7 and 8), a stage previously inaccessible to full-volume imaging[16].

In summary, we have implemented an aberration-corrected multi-focus imaging system for fast 3D imaging. Simultaneous acquisition of all focal planes eliminates the spatio-temporal ambiguity of sequentially recorded z-stacks. The resolution of the multi-focus image is that of the wide-field microscope to which it is appended, and the imaging sensitivity (light efficiency) is currently ~60% of that of the wide-field fluorescence microscope. Field of view and acquisition-rate are determined by camera chip size and read-out speed. Thanks to its ease of use, compatibility with standard microscopes and excellent imaging performance, MFM should facilitate the rapid acquisition of 3D data from the level of single molecules up to small organisms.

Online Methods

Design of the MFG grating function

In a phase-only diffractive grating, the shape of the grating pattern and its etch depth constitute the grating function, and determine the energy distribution between diffractive orders. To optimize light efficiency in MFM, we wished to direct the fluorescence emission light from the sample with maximum efficiency into the orders we chose to image. Furthermore, light should be distributed evenly between these orders, so that a minimal exposure time can be used to record each multi-focus image while still getting sufficient signal in each plane, and to obtain an even signal throughout the 3D image.

The amplitude point spread function (PSF) of an imaging system can be calculated as the square of the Fourier transform of the pupil function[7,18]. In MFM the pupil function is modified by the phase-shift of the MFG, which we can model with a matrix G. The altered PSF of an aberration-free microscope where the MFG is inserted in the Fourier plane is then given by the square of the Fourier transform of the matrix G. We have used this property to optimize the grating function (Fig. 1d) as described below. (Similar algorithms have been used before to tailor the PSF of imaging systems[17].)

We create a matrix T as the target function, representing the multi-focus PSF we desire. For a system with nine planes, the desired PSF consists of 3 × 3 points of maximum brightness and evenness. A starting guess for the phase grating G is made in the form of a matrix of randomly distributed black (−1) and white (+1) pixels representing phase shifts 0 and π, respectively. The matrix G is Fourier transformed using the command fft2 (which executes a fast 2D Fourier transform[18]) yielding the complex-valued PSF* which is squared to yield the real-valued, amplitude PSF[7]. The algorithm now randomly selects a pixel in G and flips it to the opposite phase. This matrix is Fourier transformed, yielding a new PSF. The two PSFs are compared to the target function T and the pattern that gives a PSF with a better resemblance to T is kept as the new matrix G. The algorithm steps through all the pixels in G in a randomized fashion and flips them if this gives better resemblance to the target function. It terminates when it has reached a local minimum – when no pixels are flipped during a round of stepping through all the pixels of the matrix. The algorithm is thus not guaranteed to converge to an optimal solution, but requires some trial-and error from the user to nd a good pattern. Using this algorithm we arrived at a pattern that produces a PSFs of 3 × 3 = 9 points. A similar pattern was previously described as an example of an optimally efficient pattern for this type of grating[6]. Calculated efficiency of our version of the MFG pattern was 67%, and measured efficiency of our MFG based on this pattern was measured to be 65% with evenness between focal planes. We also created a pattern producing 5 × 5 = 25 points with efficiency 78%. Grating functions and PSFs are illustrated in Supplementary Fig. 1.

Aberration-corrected refocusing

In a standard wide-field microscope, the nominal focal plane of the sample (z = 0) is conjugated, via the objective and the tube lens, to the plane of the camera. The principle of MFM is to simultaneously focus the light originating from the in-focus plane and a set of out-of-focus planes onto the camera, forming an instant focal series (the multi-focus image). Light from a point-source in the nominal focal plane has a at wavefront in Fourier space, while wavefronts from defocused sources have a curvature, the defocus wavefront-error δφ(z) (Supplementary Fig. 2a). To form an image in the nominal focal plane of the out-of-focus source, this wave-front error must be reversed so that the out-of-focus wavefront is at before it hits the final focusing lens. This task is performed by the MFG, positioned in the secondary Fourier plane. The primary Fourier plane is the objective pupil plane, which is situated inside the objective itself. We therefore use the tube lens and first relay lens, f1, to form a secondary Fourier plane, conjugate to the objective pupil (Fig 1a).

We start by computing, for each point (x) in the pupil plane, the out-of-focus phase error δφ(z) of the wavefront from a point source located on the optical axis in a plane at defocus z from the nominal focal plane[2]. We let denote the radial coordinate position in the pupil, and ρ = r / R the normalized radial coordinate, where R = NA × f is the pupil radius and f the focal length of the objective. The numerical aperture (NA) is de ned as NA = n × sin(α) where n is the diffractive index of the objective immersion medium (and sample) and α the highest incident ray angle collected by the objective. We also use the free-space wave-number k = 2π / λ (λ is the sample fluorescence emission wavelength). As illustrated in (Supplementary Fig. 2b), assuming radial symmetry and using polar coordinates, a ray entering the objective at an angle θ (θ < α) has a defocus phase error:

Using the Abbe sine condition, which states that ρ = sin(θ) / sin(α), we can relate the angle θ to radial position r in the pupil plane. We use this relation to write:

Here, we have gotten rid of the radial symmetry (in order to introduce the distortion in x and y individually) by returning to Cartesian coordinates. We can now write the equation for the defocus phase error for a plane at defocus z at every point (x) in the pupil:

A geometrical distortion δx of a periodic grating pattern (with spacing d) in the pupil plane introduces a phase shift δφ in the diffracted wavefront, which is dependent on the diffractive order m according to:

To create a multifocus image consiting of N× N planes with a constant focus step Δz, we impose a local distortion of the MFG grating pattern. (The distortion is applied to a periodic pattern with spacing d of the grating function in Fig. 1d.) To create a proper refocus, the distortion is applied stronger by a factor N in one direction:

This gives us a total focus shift in each diffractive order (m) of Δz × (m × m). For example, with N= 3 the order (m) = (+1,+1) gets focus shifted by 2 z× (1+ 3×1)= 2 z× 4 and the zeroth order (0,0) gets no focus shift.

Chromatic correction

After the MFG, each horizontal and vertical diffractive order has a dispersion δλ / d (where d is the average grating period the MFG pattern and δλ the wavelength bandwidth imaged). Each diagonal order has a dispersion due to geometry. If left uncorrected the dispersion would give rise to a severe outward smear in the image (Supplementary Fig. 3a). The CCG is placed at a distance from the MFG so that each order goes through its own designated panel. In the individual blazed grating panels, >95% of the energy is distributed into the −1 (minus one) diffractive order (Supplementary Fig. 3b). To reverse the MFG dispersion the CCG should therefore have a period equal to that of the MFG (d = d). However, if used alone, the CCG would completely oppose the image-separating action of the MFG and the diffractive orders would recombine to form a single image in the center of the sensor (Supplementary Fig. 3c). We have therefore added, after the CCG, a nine-faceted refractive prism, which maintains image separation by de ecting the orders while they are still separated (Supplementary Fig. 3d). The prism has a dispersive power Dprism = (nsilica − 1) × ξ, where nsilica is the refractive index of the UV-grade fused silica prism and ξ is the prism angle[7]. The period of the CCG is therefore adjusted by d to also remove the effect of the dispersive power of the prism, so that d = d+d in the side panels and in the corner panels. The light-efficiency of the chromatic correction module is determined by the efficiency of the blazed grating (better than 95%) and the transmittance of the (AR-coated) prism (also better than 95%). In summary, we have designed a chromatic correction scheme which is effective, light-efficient and easy to incorporate in a straight beam path between the MFG and the second relay lens, and can be used to image fluorophores from the entire visible spectrum.

Data acquisition

Cells were illuminated in epi uorescence with laser light. For green uorescent protein excitation we used a 488 nm laser (488 Sapphire, Coherent) with intensity <40–80 W cm−2. For experiments with TMR dyes, we used a 561 nm laser (MPB Lasertech) of excitation intensity <1 kW cm−2. In both cases, we used a double-band dichroic mirror (Di01-R488/561, Semrock). We used a 100x Nikon oil immersion objective with NA = 1.4 for yeast and RNA Polymerase imaging, and a 60x Nikon water immersion objective for C. elegans imaging. In the emission pathway, before the second relay lens, the uorescent light was split onto two cameras using a dichroic mirror (custom made by Chroma Technology) with cut-off at 570 nm. Emission lters centered at 520 nm (FF01-520/35, Semrock) and 617 nm (FF01 617/70, Semrock) were placed in front of the cameras in the green and red channel, respectively. For all experiments we used back-illuminated EMCCD cameras (Andor Technology). For the yeast and RNA Polymerase II imaging we used Ixon3-DU897E cameras and for the C. elegans imaging we used the iXon-888 camera, which has lower read-out speed but a larger chip size. Further descriptions of the biological imaging experiments are available in Supplementary Notes 2, 3 and 4.

Data processing

To reconstruct the 3D imaging volume from the multi-focus image, the nine focal planes were aligned on top of one another using calibration data obtained with 200-nm fluorescent beads (Invitrogen) mounted on a coverslip. We recorded a focal series by scanning the bead sample along the z-axis using a piezoelectric stage (nanoX-200 from PiezosystemJena) (Fig. 1g and h). During this process, the beads are successively focused in each of the nine sub-images. We then performed a 2D Gaussian fit on the z-projection of each focal plane to locate the beads in the different sub-images. Using rotation, translation and stretching of the sub-images, we determined a transformation matrix enabling the focal planes to be superimposed with an accuracy <10 nm. The transformation matrix was subsequently used on the biological data to reconstruct the 3D volume for each time frame. The calibration data was also used to estimate and correct the small differences in transmitted intensities between images. For 3D particle tracking we used the u-track algorithm[19] (an open source, multiple-particle tracking software).