Quantitative sectioning and noise analysis for structured illumination microscopy.

Structured illumination (SI) has long been regarded as a nonquantitative technique for obtaining sectioned microscopic images. Its lack of quantitative results has restricted the use of SI sectioning to qualitative imaging experiments, and has also limited researchers' ability to compare SI against competing sectioning methods such as confocal microscopy. We show how to modify the standard SI sectioning algorithm to make the technique quantitative, and provide formulas for calculating the noise in the sectioned images. The results indicate that, for an illumination source providing the same spatially-integrated photon flux at the object plane, and for the same effective slice thicknesses, SI sectioning can provide higher SNR images than confocal microscopy for an equivalent setup when the modulation contrast exceeds about 0.09.

1. Introduction

Structured illumination (SI) is an optical sectioning technique compatible with widefield imaging microscopy, and has been shown to provide a depth resolution comparable to confocal microscopy [1]. Since its invention [2], SI microscopy has been widely used as a sectioning tool in bioimaging research, both at the cellular level — such as in 3D imaging of cellular nuclear periphery [3], cellular fenestrations [4], tubulin and kinesin dynamics [5] and at the tissue level — such as in imaging of autofluorescence aggregation in human eye [6], zebrafish development [7], rat colonic mucosa [8]. In addition, SI has also been used as a super-resolution technique to break the diffraction limit [5, 9–11]. SI thus has the ability to maintain the high light collection capability of widefield imaging [12,13] while also removing out-of-plane light. It has, however, been criticized as being a non-quantitative technique, and for producing noisy data in comparison to the sectioned images derived from confocal microscopes. The analysis below shows that SI can easily be made quantitative by properly scaling the standard sectioning algorithm, and we also provide analytical expressions for the resulting noise in SI-sectioned images. Although Somekh et al. [14, 15] provide numerical simulations of the noise properties of SI-sectioned images, they use a non-quantitative algorithm and do not include the effects of out-of-focus light on the noise.

Quantitative sectioned images allow one to perform photon counting as if the regions above and below the sectioned layer were not present. While the resulting photon number estimate will be noisier than would be the case for imaging the slice without out-of-focus layers present, the mean value of the correctly scaled algorithm will equal to the mean photon count one would obtain with a standard widefield microscope. This permits researchers to use standard methods [16] of correcting for the objective lens’ numerical aperture, optical transmission, and detector quantum efficiency to determine photon counts at the sectioned plane relative to the number of photoelectrons detected at the sensor plane. Allowing quantitative data to be obtained with SI sectioning thus gives researchers the ability to perform measurements of radiance, absolute reflectance, fluorophore quantum yield, and absolute fluorophore concentration within volumetric media [17, 18].

Finally, the analytical formulas for noise also enable us to roughly define an operational range for the modulation contrast at the section plane, such that for any contrast above this value one can expect SI to provide higher SNR data than confocal microscopy. For any contrast below this value, confocal imaging will out-perform SI.

2. Sectioning algorithm

The general approach in SI is to illuminate the object with a sinusoidal illumination pattern of the form [2] at each of three spatial phases ϕ1 = 0, ϕ2 = 2π/3, and ϕ3 = 4π/3. Although other structured forms are possible [19], this one is particularly easy to implement. The quantity m is the modulation contrast (a number varying from 0 to 1), and ν is the modulation spatial frequency. The factor of 1/2 placed in front, not present in previous work, is used here in order to represent the fact that half of the illumination light is absorbed or reflected by the grid placed in the illumination path. If we take the limit m → 0, we obtain standard widefield illumination of half the intensity that one would obtain without the grid in place. Note that s represents a normalized illumination amplitude, ranging from 0 to 1.

Ignoring the effects of optical blurring, the resulting modulated images g(x, y) are given by for a planar object distribution f(x,y) and out-of-focus light d(x,y), both scaled to what one would obtain with standard widefield imaging. For fluorescence imaging, the absolute brightness f of the object contains the illumination irradiance I, the fluorophore quantum yield q, and a factor Ω resulting from integrating the angular distribution of fluorescence emission over the numerical aperture of the imaging optics: ffluor = IqΩ. For brightfield imaging, f is simply the illumination I multiplied by the object reflectance R: fbright = IR. Thus, the expression for g is valid for the cases of both fluorescence imaging and brightfield imaging, but with a subtle difference in what f means for each case.

In order to obtain an optically sectioned image i(x,y) at the focal plane, the most common algorithm used is [1, 2, 8, 20–31] based on square law detection. Alternative illumination patterns allow for different processing approaches [19]. SI for superresolution, for example, relies on the Moire effect to detect light emitted outside the conventional bandwidth limit.

Since the algorithm (3) operates on each pixel independently, we have dropped the spatial arguments (x,y) as unnecessary. (These can be added back into each equation at any point.) Inserting Eqs. (1) into (2) and applying trigonometric identities, we obtain the result Thus, the sectioned image is a copy of the object distribution, as we expect, but scaled by the factor . The quantitatively-scaled algorithm is obtained by multiplying Eq. (3) by the inverse of this factor: Note that the algorithm assumes that the out-of-focus light d(x,y) does not change with a shift in the illumination pattern. A consequence of this result is that in order to obtain quantitative results for the sectioned image, one must estimate the modulation contrast m. A further assumption required is that of linearity, which in fluorescence imaging is limited to weakly fluorescent structures [1].

The scale factor in front of the square root differs from that given in previous studies. The factor of 2 in the numerator appears as a result of the 1/2 scaling introduced into our definition of s(x, y) and thus is new. All previous authors have also assumed ideal modulation (m = 1). This is an assumption which introduces a large error into the quantitative result. Moreover, since the modulation m(x,y) is in general spatially varying, the error introduced is generally not a simple scalar factor for the whole image. As a whole, the literature shows wide disagreement over the appropriate scale factor to place in front of the square root. Refs [20,22,26,28,32,33] use , which is appropriate when m = 1 and the factor of 1/2 in s(x,y) is not used. Refs [2, 23–25,27,29] use a scale factor of 1, which is the most appropriate choice for a non-quantitative approach, while other authors use alternative factors such as [8], [1, 31], or [21] without explanation.

In practice, one finds that even for ideal samples m cannot achieve the maximum value of 1. The modulation contrast, however, remains excellent (m > 0.5) in thin samples in which the sectioned plane is taken near the surface, but poor (m < 0.1) in dense tissue samples (in which multiple scattering is present) and in deeper layers of thinly scattering media.

3. Widefield image algorithm

In addition to the sectioned image algorithm (5), it is well known that one can form a widefield image representation i(x, y) from the modulated images by (Our scale factor differs from that of previous authors due to the factor of 1/2 used in the definition of s(x,y).) Once again inserting the formulas for the illumination and modulated images, Eqs. (1) and (2), we obtain The widefield image is a simple sum of the sectioned plane and the out-of-focus contribution.

For quantitative work, we also want to estimate the variance of the widefield image, for which we insert Eqs. (1), (2), and (6) into the standard variance formula and solve. Here the angle brackets 〈·〉 represent an expectation value. The first term in the variance formula is Here we have assumed that d and f are independent of one another, so that terms such as 〈d + f〉 = 〈d〉 + 〈f〉 and 〈df〉 = 〈d〉〈f〉. Because the s represent a normalized illumination distribution, the stochastic properties of the system are present only within the out-of-focus light d and the slice’s light distribution f, and not in the illumination s.

The second moment thus separates into four terms, each of which can be considered separately. Using trigonometric identities, we obtain for the modulation factor in each term giving the result where we have used 〈d2〉 = 〈d〉2 + var(d).

The second term in the variance formula is easily obtained from Eq. (7) as Putting the two results together produces Just as the mean value of the widefield image is a simple sum of the planar slice and the out-of-focus light, 〈i〉 = 〈d〉 + 〈f〉, the variance of the widefield image is also a simple sum of the component variances.

4. Variance and SNR of sectioned images

Next we can try to follow the same procedure for the sectioning algorithm (5) to obtain the variance of the sectioned image, var (i) = 〈i2〉 − 〈i〉2. The second term in the variance formula is easily obtained from Eqs. (4) and (5) as 〈i〉2 = 〈f〉2. The first term can be obtained by inserting Eq. (6) to give The next step is to substitute Eqs. (1) and (2) into this formula, but here one must be careful. Each modulated image g provides different samples d and f of the stochastic out-of-focus and slice light distribution, such that terms like will be nonzero. That is, since d1 is independent of d2, we can write 〈d1d2〉 = 〈d1〉〈d2〉, whereas . Once all cross-terms are eliminated from inside an expectation value, one can then take 〈d〉 → 〈d〉 and var(d) = var(d), and likewise for the f as well. Thus, the first quadratic term inside the expectation value of Eq. (13) is Letting all 〈d〉 → 〈d〉 and , we have Doing this also for the second and third squared terms in Eq. (13) and combining gives where we have also used the results of Eqs. (10) and (11). Incorporating this result into the variance formula obtains The variance of the sectioned image is thus dependent on the out-of-focus contribution in addition to the variance of f. (Recall that f represents the light obtained from a single standard widefield image of just the planar slice itself.) Both terms contain a dependence on the modulation contrast, so that as the modulation approaches zero (m → 0), the variance in the sectioned image increases without bound, as we should expect.

The theoretical expression for the sectioned image variance Eq. (14) indicates that when the illumination produces ideal modulation contrast at the slice plane (m = 1), the sectioning algorithm amplifies noise in the slice image by a factor of (in standard deviation) relative to the stochastic noise in f. For most cases, a large out-of-focus contribution is present, and this not only reduces the modulation contrast but adds to the noise as well. Then m becomes small, and in this regime the variance approximates to so that the signal-to-noise ratio of the sectioned image in the weak modulation regime is reduced by a factor relative to that of a standard widefield image,

5. Estimating the modulation contrast

The quantitative sectioning algorithm (5) requires knowing the modulation contrast m in order to properly scale the result. This value is generally not known a priori and so must be estimated, but one can use the modulated images themselves to provide the estimated value m̂. For each of the three modulated images, we obtain a “modulation map” by normalizing the modulated images using the widefield algorithm as Since μ can also be written as , subtracting 1/2 from μ produces a result proportional to m, so that and an estimate of m is thus given by This suggests that, for high SNR data, one can calculate the sectioned image pixel-by-pixel by combining Eqs. (5) and (15) to give

As an example, we measured the three modulated images g1, g2, and g3 on a fluorescent bead sample and use the algorithm in Eq. (15) to obtain the estimated modulation m̂ (x,y) at every pixel in the image. Our experimental setup (see Sec. 6.1) achieves approximately uniform modulation across the image, and so if we first threshold the image iw to prevent noisy pixels from skewing the estimate, we obtain a histogram of m̂ across the image, as shown in Fig. 1. The histogram suggests that the pixel-by-pixel estimate of m will be quite noisy, so that a much more accurate estimate can be achieved by averaging m̂ across the image. Or, if there is a significant contribution of outliers, one can use the histogram median as a more robust estimate. For Fig. 1, the mean and median values are m̂ = 0.48 and 0.49 respectively. We can note, however, that taking the mean or median are only valid for homogeneous samples.

Fig. 1

A histogram of the estimated modulation contrast, m̂, obtained from (15).

From a theoretical standpoint, the roughly Gaussian shape to the histogram is expected, but the long tail at the lower values of m̂ is not, and may be the result of some beads aggregating together to create a thicker layer at some locations within the image. If the thickness exceeds the sectioning depth, then the modulation contrast will drop.

While Fig. 1 shows that the median modulation contrast can be accurately estimated from a single image, the use of Eq. (15) to estimate m(x,y) requires exceedingly high signal-to-noise ratio images in order to achieve a reasonable accuracy for every pixel in the image. Its practical use, therefore, requires collecting a large number of photons (perhaps by summing a sequence of static images) or some kind of spatial processing in order to reduce the effects of noise.

6. Experimental results

In order to test our theoretical results, we conducted several experiments on a Zeiss Axio Imager Z1 microscope equipped with an Apotome module, a Zeiss AxioCam MRm monochromatic camera (1388 × 1040 pixels), and an HB-100 mercury lamp illumination source. The objective lens used for all of the experiments is a Zeiss Plan-Apochromatic 20× objective (NA = 0.8). In order compare measurements against theory, we use the ratio r of measured noise to the estimated shot noise. Whereas the measured noise is obtained by taking the standard deviation of a sequence of 1000 measurements, the photon shot noise standard deviation σp is estimated by taking the square root of the mean number of photons collected. For a standard widefield measurement, this number should be close to 1, but for SI-sectioned images the noise is larger than one would expect from the shot noise alone, so that the theory predicts r > 1.

The first step of the experiment involves measuring the camera gain in order to scale digital counts to detected photelectrons. This involves imaging a uniformly illuminated field (created by Köhler illumination) at the microscope sample stage with different illumination intensities. To remove the effects of pixel response nonuniformity, we implemented the following procedures: [34] The quantities fc and σc here indicate the measured intensity and standard deviation in units of counts and not photons. The resulting measured signal vs. variance at different illumination intensities is shown in Fig. 2, and gives an estimated gain g = 4.1 photons/count.

Fig. 2

Signal vs. variance measurement, showing experimental data (dots) and the fitted line. The line slope of 4.1 gives an estimate for the gain of g = 4.1 photons/count.

At each illumination intensity, two wide field images I1 and I2 are acquired.

The standard deviation σc is calculated for a 200 × 200 pixel area in the difference image I1 − I2.

The signal variance is calculated by . The scaling factor of 2 accounts for the increased noise due to the image subtraction operation in Step 2.

In order to provide a baseline reference for later SI noise measurements, we imaged a microscope slide containing a sparse layer of fluorescent beads (Molecular Probes Fluosphere F8853, peak emission at 515 nm, 2 μm diameter) in wide-field mode (i.e. without the structured grid placed in the illumination path). To prepare a uniformly distributed sample, the fluorescent beads were suspended by vortex mixing and sonicated. The suspensions was then dropped onto a microscope slide and sealed with a cover slip. A total of 1000 images of the sample were acquired in a time sequenced experiment. The measured mean fluorescent intensity 〈f〉 of the fluorescent beads is 4.33 × 104 counts (obtained by summing all pixels at the bead location), and the standard deviation σ of fluorescent intensity is 112 counts. Thus the ratio of measured noise to the photon shot noise is: Since r is very close to 1, we can say that the imaging system is shot-noise limited.

6.1. Sectioned imaging of 2 μm fluorescent beads without out-of-focus light

We first measured the axial PSF of the SI-sectioned measurements by imaging sub-resolution green fluorescent nanoparticles (175 nm diameter spheres, from Invitrogen), using the software-recommended VL grid (17.5 lines/mm) on the Apotome. The FWHM of resulting measured axial PSF is 3.6 μm, indicating that our fluorescent bead sample (peak emission at 515 nm, 2 μm diameter) is sufficiently thin that no out-of-focus light will be present. For the quantitative algorithm, we estimate the modulation contrast for this setup using the data shown in Fig. 1, giving m̂ = 0.49 from the median of the distribution. A total of 1000 sectioned images were acquired in a time sequenced experiment. The mean intensity 〈f〉and its standard deviation σm are calculated for 5 different beads, with results shown in Table 1. The ratio of measured noise to the photon noise is calculated as: On the other hand, the theoretical value of the ratio r, assuming Poisson noise, is obtained by The theoretical value of 2.49 thus closely corresponds to the measured mean value of 2.41 given in Table 1. That is, use of the sectioning algorithm with the three modulated images on the (planar) sample has reduced the SNR by a factor of 2.41 from that of a single (unmodulated) widefield image.

Table 1

Measurements on a sample of five different fluorescent beads without out-of-focus light. These are the measured mean widefield intensity 〈f〉 and its standard deviation σm(f), the calculated widefield intensity 〈iw〉 and its standard deviation σm(iw), and the corresponding ratios r obtained with Eq. (17). The modulation contrast m used to calculate 〈f〉 is estimated using Eq. (15).

Sectioned image				Calculated widefield image
Bead	〈f〉 (cts)	σm(f) (cts)	r	〈iw〉 (cts)	σm(iw) (cts)	r
1	5.03 × 104	279	2.53	4.99 × 104	114	1.03
2	6.29 × 104	292	2.35	5.51 × 104	122	1.06
3	4.61 × 104	259	2.44	4.59 × 104	115	1.08
4	5.04 × 104	277	2.50	4.93 × 104	110	1.00
5	6.17 × 104	276	2.24	5.60 × 104	119	1.02
		mean:	2.41	mean:		1.04

The calculated widefield image iw of the same sample is obtained by algorithm (6). The mean intensity 〈iw〉, standard deviation σm and ratio r are calculated for the same five beads, with results shown in Table 1. The measured value of the noise ratio r = 1.04 closely corresponds to the theoretical result of r = 1 obtained from Eq. (12).

6.2. Sectioned imaging of 6 μm fluorescent beads containing out-of-focus light

In order to measure noise amplification in SI in the case when out-of-focus light is present, we imaged a sample of 6 μm diameter green fluorescent beads. Since the sectioning thickness of the Apotome in this setup is 3.6 μm, there will be some out-of-focus light present. A total of 1000 sectioned images were acquired in a time sequenced experiment. The mean intensity 〈f〉 and its standard deviation σm are calculated for five different beads, with results shown in Table 2. The theoretical value of the ratio r, which we write as r̂, is given by If we use the relation 〈iw〉 = 〈d〉 + 〈f〉, let m̂ = 0.49, and substitute the mean 〈iw〉 with its measured value iw, we have The resulting theoretical r̂ values for the five beads are shown in Table 2, and indicate a close correspondence with the experimentally measured value for r. Note that the variation in r̂ among the five beads selected may be due to a variation in the relative amount of fluorescence emitted by the bead from within the sectioned plane to that emitted from outside the sectioned plane.

Table 2

The mean intensity of wide-field images, 〈iw〉, and standard deviation σm of five different fluorescent beads, obtained from experiment and theory.

Experiment				Theory
Bead	〈f〉 (cts)	σm (cts)	r	iw (cts)	r̂
1	1.70 × 105	836	4.09	5.01 × 105	4.11
2	1.20 × 105	669	3.90	4.13 × 105	4.38
3	10.29 × 104	511	3.24	2.32 × 105	3.56
4	1.89 × 105	750	3.50	5.04 × 105	3.85
5	9.05 × 104	526	3.54	2.80 × 105	4.15
		mean:	3.66	mean:	4.01

7. Conclusion

Although it has often been argued that structure illumination sectioning microscopy is a non-quantitative technique, we have shown that a quantitative version of the algorithm can be obtained by adding a proper scaling factor. Quantitative scaling does require that one estimate the modulation contrast m, and this adds an extra step of complexity, but Eq. (15) provides a simple means of obtaining such an estimate. A consequence of ignoring this scaling factor, as the sectioning algorithms have up to this point, is that in z-stack volumetric images (x,y,z) the deeper layers will appear artificially darkened. A result of the SI sectioning approach, however, is that since it removes out-of-focus light from the sectioned image after detection, it suffers from the shot noise of both the section image and all out-of-focus planes. While this has long been known, little has been known about the quantitative correlation between the noise amplification in SI microscopy and the out-of-focus light or other imaging parameters.

The theoretical analysis given above has made no assumptions about the properties of the noise other than to assume that the variance is the primary quantity of interest, and thus the results remain valid across all noise regimes (read noise limited, shot noise limited, etc.).

The noise amplification indicated by the variance result may be taken as an argument that SI-sectioning is a poor substitute for confocal sectioning due to the loss in SNR. But this is not the whole story. The compatibility of SI with widefield imaging also allows orders of magnitude greater light throughput than that achievable by confocal microscopy, such that one can use lower-intensity light sources and still obtain 100–200× increases in photon collection above that of scanning laser illumination [12,13]. In this case, taking 150 as a representative value for increased light collection, SI sectioning can provide better SNR than confocal sectioning when

An additional advantage SI microscopy has is the ability to reject any residual DC light, such as that generated by stray light or reflections within the optical system, though this comes at an SNR penalty. Whether SI-sectioning or confocal sectioning produces better SNR images is dependent on the microscope setup and the object under analysis, but our theoretical results provide support for the common empirical observation that SI-sectioning gives lower quality results when imaging deep within tissue.