Extending single-molecule microscopy using optical Fourier processing.

This article surveys the recent application of optical Fourier processing to the long-established but still expanding field of single-molecule imaging and microscopy. A variety of single-molecule studies can benefit from the additional image information that can be obtained by modulating the Fourier, or pupil, plane of a widefield microscope. After briefly reviewing several current applications, we present a comprehensive and computationally efficient theoretical model for simulating single-molecule fluorescence as it propagates through an imaging system. Furthermore, we describe how phase/amplitude-modulating optics inserted in the imaging pathway may be modeled, especially at the Fourier plane. Finally, we discuss selected recent applications of Fourier processing methods to measure the orientation, depth, and rotational mobility of single fluorescent molecules.

Introduction

Even though the detection and spectroscopy of single molecules in condensed phases was first demonstrated 25 years ago,[1] the field has continued to expand in physics, chemistry, biology, and materials science.[2] Critical to the progress in understanding single-molecule behaviors has been theoretical insight to define underlying mechanisms, such as spectral diffusion and other dynamical processes in solids, for example.[3−6] Although early work at low temperatures relied on the power of high-resolution spectroscopy, the current focus on room temperature measurements and biological applications provides a continuing impetus to extract more and more information from the emitted photons. In particular, the optical field emitted by a single molecule in the far-field is given by the emission pattern of an oscillating electric dipole, which itself contains information that can be extracted by clever design of imaging systems. This paper describes how a careful examination of the electric field patterns emitted by a single molecule as well as Fourier plane modulation of these patterns can be utilized to obtain much deeper insight about precise physical properties of the single molecule.

Optical Fourier processing is a powerful tool that has been leveraged throughout the many disciplines of microscopy to tease information from a sample.[7] Filtering the individual spatial Fourier transform components of an image allow experimental information that is not directly observable from a conventional image to become more readily accessible. Currently, implementations of Fourier filtering within the imaging and illumination pathways of a microscope have permitted researchers to better quantify features such as the optical phase, and three-dimensional structure of a specimen.[8,9] Furthermore, judicious Fourier processing permits images to be enhanced, and optical aberrations to be mitigated.[10] The fundamental technique is quite simple: By placing a lens one focal length behind a spatially coherent light source (such as a laser, a fluorescent molecule (which is coherent with itself), or a star viewed in the night sky), a scaled Fourier transform of the electric field associated with the source will be projected onto a plane one focal length behind the lens. The individual spatial frequencies associated with the image of the source may then be independently adjusted by placing a transmissive or reflective mask of varying opacity (to perform amplitude modulation), or varying refractive index or thickness (to perform phase modulation). A modulated image of the optical signal may then be obtained by performing an additional Fourier transform, i.e., an inverse Fourier transform and a reflection of image coordinates, using one more lens.

The advent of single-molecule fluorescence imaging[11−13] has provided researchers with unparalleled insight into the nanoscale structure and organization of biological systems.[14,15] Using only a wide-field optical microscope, it is possible to acquire images of individual molecules on a camera sensor with single-photon detection sensitivity. To some extent, the image formed from a single molecule’s fluorescence will resemble the diffraction-limited point-spread function (PSF) of the microscope. By fitting a model function, such as a Gaussian, to recorded single-molecule images, one may infer the lateral position of a molecule with precision approaching a single nanometer, depending upon the number of photons emitted by the probe.[16,17] This technique, termed super-localization, permits the locations and movements of biomolecules within living cells to be determined.[18−20] The ability to select different single emitters in the same irradiated volume by a control variable such as spectral scanning has allowed imaging beyond the diffraction limit, thus achieving super-resolution.[21,22] In the past decade, super-resolution imaging an order of magnitude beyond the diffraction limit has expanded to room temperature studies by using various methods of actively controlling the concentration of emitting molecules and sequential imaging.[23−25] Today, using merely a widefield epifluorescence microscope, single-molecule super-resolution imaging has yielded impressive results.[14,15,26] However, with the addition of optical Fourier processing, it is possible to obtain even more information about the physical processes occurring in a specimen under observation. That is, it is possible to modify a microscope’s PSF to yield a raw (image) data set more amenable to extracting additional parameters of the molecules. For example, optical processing may be used to infer the axial depths of individual molecules within a sample, permitting super-localization[27] and thus super-resolution[28−30] to be extended into three dimensions. Furthermore, the orientation of a molecule with respect to the microscope objective lens may be more readily determined.[31] The potential for future innovation is tremendous. It is our hope that this article will inspire other researchers to further develop and refine Fourier processing techniques and apply their innovations to single-molecule imaging. There are undoubtedly new methods and data analysis algorithms waiting to be discovered.

This article has been organized as follows: In section 2, we survey some of the latest Fourier processing techniques that are most applicable to single-molecule imaging. In section 3, we provide the rigorous theoretical background necessary to computationally simulate the image of a fluorescent molecule on a camera sensor and describe how phase or amplitude modulation applied at the Fourier plane may be incorporated into the simulation. Particular attention is paid to modeling the features of a high-NA optical system, as well as polarization effects that are normally not considered for Fourier processing applications, yet are required to accurately model single-molecule emission. In section 4 we present a phase mask design and experimental apparatus recently developed by our laboratory specifically tailored for acquiring single-molecule orientation measurements. In section 5, we adapt our phase mask design to super-localize molecules in three dimensions, while simultaneously collecting data pertaining to molecular orientation and rotational mobility.

Background

To properly explain the techniques covered in this section, it is necessary to introduce a modicum of specialized vocabulary and mathematical formalism. For convenience it is best to have in mind a widefield fluorescence microscope which illuminates a region of a sample and records the image of the emitted fluorescence on a two-dimensional detector. The additional experimental apparatus used for optical Fourier processing is often termed a 4f imaging system and is sketched schematically in Figure 1. The plane at which the Fourier transformed electric field is present, one focal length between the two lenses of the 4f system, will also be assumed to be conjugate to the back focal plane or pupil plane of the microscope. That is, even though the back focal plane is physically located at a limiting pupil (aperture) one focal length behind the objective lens of a conventional microscope, the 4f system recreates a scaled image of this plane onto a region of space outside the microscope where one can place a phase or amplitude mask responsible for modulating the incident light field. Hence, from a modeling standpoint, the mask may be assumed to have been placed physically inside the microscope. Mathematically, the 4f system can be described using the following formula:[7]In eq 1, Ein(x°,y°) and Eout(x′,y′) are vectorial quantities denoting the electric fields at the input and output planes of the 4f system, parametrized by Cartesian spatial coordinates {x°, y°} and {x′, y′}. Ebfp(x,y) is the Fourier transformed input electric field, scaled by the focal length f4f of the lenses used to construct the 4f system. This field is equivalent to the electric field present at the microscope’s back focal plane, after a scaling of spatial coordinates. k = 2π/λ is the wavenumber, λ is the wavelength, and the refractive index of the medium surrounding the 4f system is assumed to be 1. C is a scaling factor that also incorporates the overall wavefront curvature. However, this term will not affect the relative amplitudes of the fields at the output of the 4f system, and therefore it may generally be ignored. Finally, Ψ(x,y) is a complex-valued scalar that denotes the amplitude and phase modulation imparted by a mask placed one focal length between the two lenses of the 4f system. Although amplitude modulation masks have found numerous applications in other fields, use of partially opaque optics is generally frowned upon in the context of single-molecule fluorescence microscopy, as one cannot easily afford to waste precious photons. Hence for the remainder of this article, we will assume that the modulation function Ψ(x,y) has modulus 1 and is thus a pure phase mask:where ψ(x,y) is the desired phase function applied at the back focal plane. The features of Ebfp(x,y) may be “stretched” or “compressed” by choosing a different focal length, f4f′, for the lenses used to construct the 4f system. That is, the electric field at the back focal plane, Ebfp′(x,y), associated with a lens of focal length f4f′ can be computed aswhere the leading amplitude-scaling term enforces conservation of energy. Hence, given a phase mask of fixed dimensions, it is feasible to choose lenses to appropriately scale the spatial extent of the impinging electric field. It is practically expedient to construct a 4f system from relatively weak lenses of focal lengths ∼150 mm, as this will make the dimensions of the lenses themselves small relative to the overall extent of the optical system, thus minimizing the impact of optical aberrations. Furthermore, our laboratory generally uses achromatic doublets for constructing a 4f system to ensure that the focal length, and hence the location of the back focal plane does not change significantly as a function of wavelength.

Figure 1

Schematic diagram of a 4f optical processing system. In a conventional microscope, light is collected by an objective lens, collimated, and relayed through the back focal plane of the objective. (The z direction is always assumed to lie parallel to the optical axis.) A tube lens focuses the collected light into an image at the “intermediate” image plane. The 4f system (red box) is inserted a distance f4f behind the intermediate image plane. The Fourier transformed electric field, Ebfp(x′,y′) (a scaled image of the back focal plane), is projected onto a phase mask, and the phase modulated field is then focused into an image on a detector by a second lens. The precise scaling of the back focal plane image incident upon the phase mask will depend upon the numerical aperture of the objective, and the magnification of the objective/tube lens pair (calculated from fobj and ftube).

Practically, phase masks with a desired response ψ(x,y) may be implemented by varying the thickness of a transparent glass or polymer dielectric using photolithographic fabrication techniques.[32,33] Alternatively, if one wishes to dynamically alter ψ(x,y) over the course of an experiment (for example, to correct for sample-specific optical aberrations), high-diffraction-efficiency liquid crystal on silicon spatial light modulators (SLMs) are commercially available.[34] These devices are typically composed of individual cells, or pixels, containing a liquid crystal material embedded above a reflective surface (transmissive SLMs are also available but suffer from reduced photon-efficiency). In response to an applied voltage on each pixel, the liquid crystal effectively changes refractive index and hence varies the phase-lag accrued by incident light. SLMs are an excellent choice for generating phase masks with discontinuities, as the refractive index of a given pixel may be adjusted independently of its neighbors. Recently, SLMs have been reported to achieve >90% reflectivity.[27] However, these devices typically modulate only one polarization of incident light. Hence, care must be taken to either reject light of the incorrect polarization or correctly rotate its polarization before incidence on the SLM. If the phase function ψ(x,y) possesses no fine discontinuities, deformable mirrors (which currently are only available with limited numbers of pixels for reasonable cost) may be used to encode the desired modulation. Recent developments in microelectromechanical system (MEMS) fabrication techniques have enabled the emergence of deformable mirrors featuring a single continuous metallic surface, which may be bent into a desired shape using an array of electro-static (or magnetic) actuators.[35] Because the individual actuators of deformable mirrors can travel distances of ∼10 μm, these devices often have a greater range of phase modulation than SLMs. Furthermore, their reflectivity is superior to that of SLMs, leading to better overall collection of emitted photons. Hence, deformable mirrors are often the method of choice for correcting optical aberrations,[36] which are generally smoothly varying functions. On the other hand, SLMs are used to impart more exotic phase masks that contain rapidly varying features.

Microscopists have developed a menagerie of phase masks for single-molecule applications. The experimenter’s responsibility is to decide upon the measurements that need to be acquired and then choose the phase mask that is optimal. In this section, we give a brief overview of the designs recently reported in the literature, and the scope of their application. By far, the most popular single-molecule application of a phase mask is to precisely determine the z-positions (depths) of molecules within a sample. Depth information may be trivially obtained simply by inserting a cylindrical lens in the imaging pathway.[37,38] Use of such an optic will make the microscope’s PSF appear elliptical, or astigmatic. That is, the microscope will exhibit two different focal planes along the x- or y-axis at different z-positions. The z-position of a molecule is recovered as follows: An asymmetric 2D Gaussian function is fit to the image of an emitter formed on the detector, and the widths of the fitted (elliptical) Gaussian along the x- and y-axis of the detector are recorded. The z-position is then inferred from a lookup table relating the image widths along the x- and y-directions to depth, which is constructed by translating the microscope objective lens in known increments relative to a fluorescent bead.

Though the use of an astigmatic lens is cheap, simple, and effective, a number of alternatives for inferring depth have been proposed, offering considerable advantages. For example, in collaboration with researchers at the University of Colorado, Boulder, our lab has implemented a “double helix point spread function” (DH-PSF)[39,40] that modulates the PSF into a particular superposition of Gauss–Laguerre modes causing the PSF to resemble two Gaussian “lobes”, which appear to revolve about a fixed point as the emitter is translated along the microscope’s optical axis. By fitting two Gaussians to this raw data, and measuring the angle of the line formed by connecting the centers of the two lobes, depth is inferred by comparing angle measurements to a lookup table. The efficacy of the DH-PSF has been demonstrated in biological tracking applications,[27,41] as well as in super-resolution imaging experiments in living cells.[28,30]

The double helix point spread function is an attractive alternative to astigmatism, because it has superior Fisher information content,[42,43] which means that given a photon shot noise-limited image, a 3D localization can be obtained to greater precision than would be possible using the astigmatic PSF. Sophisticated image-fitting algorithms that make more effective use of raw data are also currently available.[44] Recently, related PSF designs have specifically sought to maximize the Fisher information throughout a range of microscope defocus settings.[45] Other superpositions of Gauss–Laguerre modes may cause the PSF to appear as one revolving lobe as opposed to two, thus forming a corkscrew shape.[46] A similarly behaved PSF design was also derived using spiral phase gradients (vorticies) applied in successive Fresnel zones throughout the microscope’s back focal plane.[47] This design variant may be particularly useful when one attempts to detect or compensate for spherical aberration. Furthermore, using a dual polarization configuration, a phase mask design based upon Airy beams has been developed, capable of localizing emitters with z-precision equivalent to the lateral (x/y) localization performance.[48]

Although phase masks that allow emitter depth to be measured cause a well understood z-dependent aberration to become manifest in the microscope PSF, it is sometimes preferable to induce as little change in the PSF as possible over a given range of z-positions, effectively increasing the microscope’s depth of field. A variety of extended depth of field (EDOF) designs relying on a cubic phase function[49] have been investigated. Furthermore, these designs have been characterized with regard to the effects of spherical aberration (a common occurrence in microscopy when a sample’s refractive index does not match that of the immersion medium of the objective).[50] Additionally, an expanded point information content (EPIC) phase mask has been developed that exhibits extended depth of field on one side of focus, and permits the z-position to be measured on the other side of focus.[51] Hence, these two favorable qualities in the PSF may be simultaneously realized by dividing the fluorescence emitted by the microscope into two separate imaging channels, and recording images using two sensors focused at different z-positions within the sample (biplane).[52]

The majority of applications of Fourier processing to single-molecule microscopy have emphasized either retrieving a molecule’s depth or mitigating the effects of microscope defocus or aberration. However, specialized phase masks may be used to extract other physical parameters. For example, the orientation of individual molecules is a subject of enduring interest to microscopists.[31] Fortunately, the far-field emission pattern associated with a rotationally immobile molecule is highly anisotropic, which implies that light emanating from a molecule will form a nonuniform intensity distribution upon a microscope’s back focal plane, which is a function of the molecule’s orientation relative to the imaging system. This feature may be exploited to deduce the underlying orientation of any molecule of interest.[53] Phase masks especially devoted to performing single-molecule orientation measurements, as well as simultaneously inferring position and orientation information[71] will be described in the final two sections of this paper.

Theory

To profit from the potential that optical Fourier processing provides for single-molecule imaging, it is necessary to develop an accurate theoretical framework capable of predicting the emission patterns that are acquired from an optical system. Given a phase mask design and a fluorescent molecule, we wish to simulate the image that would actually appear on a camera sensor, prior to doing an actual experiment. Although simply approximating a single-molecule image as the microscope’s PSF may be sufficient for some applications, a more robust simulation model is generally required to maximize information extracted—especially for applications involving the determination of molecular orientation. By comparing simulated images to actual data, it is then possible to draw conclusions about the sample under observation.

In this section, we present formulas that may be used to computationally generate images of a single molecule at an arbitrary depth or orientation within a sample of interest. Furthermore, we discuss the modeling considerations that may be used to accurately predict the effects of a phase mask upon the final image recorded by the optical system. Though eq 1 from the previous section tells us much of what we need to know to choose a proper phase mask for a given application, it does not tell us the precise functional form of the electric field input into the 4f system, or the field impinging upon the back focal plane. These fields can turn out to be quite exotic when one considers a single molecule as an illumination source. The derivation contains two major steps: In the first step, we must calculate the far-field emission from a molecule, in addition to modeling the high numerical-aperture (NA) objective used to collect light exiting the sample with vectorial diffraction theory. This allows us to determine the intensity distribution present at the back focal plane, which is essentially the scaled (spatial) Fourier spectrum of the intensity distribution at the image plane of the microscope. In the second step, we incorporate the use of a phase mask into our imaging model, which involves the far simpler theory of Fourier optics because the optical fields are paraxial at this point in the imaging system. Furthermore, we demonstrate some computational shortcuts to calculating image plane intensity distributions. Our method involves the use of basis functions which allow molecules of arbitrary orientation to be simulated rapidly, after some modest preliminary computation.

Calculation of the Back Focal Plane Intensity for a Single Fluorescent Molecule

Under most circumstances, the emission pattern of a single fluorescent molecule most closely resembles that of the far-fiels of an oscillating electric dipole. We begin by specifying the orientation of the transition dipole moment of a molecule using the unit vector μ̂. The orientation of the dipole is defined by an azimuthal angle Φ and a polar angle Θ, whereas shown in Figure 2a. By solving Maxwell’s equations for an impulse current source, it can be shown that the intensity distribution that appears in the “far-field”, a distance r ≫ λ from the molecule in question, resembles a torus centered along the axis specified by μ̂ (Figure 2b).[54] That is, if one were to measure the far-field intensity Iff along a single ray emanating from the molecule in a direction specified by a unit vector r̂, or by the angles {ϕ, θ}, such thatOne would findwhere η is the angle between the two vectors, r̂ and μ̂. To describe the electric fields that give rise to this intensity distribution, a convenient mathematical object exists, which is called the Green’s tensor:[55]In the above equation, n1 is the refractive index of the medium in which the molecule has been embedded. † denotes the adjoint operator. To find the electric field at any point along the surface of a sphere of radius r, simply computewhere A denotes the amplitude of the molecule’s dipole moment. By examining the expression for Gff(ϕ,θ), one may observe the following features: First, the term (I – r̂r̂†) enforces the sin2(η) intensity dependence and also ensures that the electric field traveling along a ray specified by r̂ will have no component parallel to the direction of propagation. The scalar term, ei/4πr, accounts for the fact that as the sphere defined by r grows, the total energy emitted must remain constant, leading to an attenuation of the electric field measured at a single point along the sphere’s surface. The exponential term arises due to the phase-lag incurred by a light wave traveling a distance r away from the molecule. It is also useful to make the following observation: If the molecule is moved along the optical axis a short distance d ≪ r, such that r′ ≈ r and θ′ ≈ θ (Figure 2c), then the resulting field may be calculated by augmenting eq 8 with an additional phase factor:Equation 9 is helpful when the effect of microscope defocus is considered, which may be modeled as an axial displacement of the microscope’s focal plane from the molecule of interest.

Figure 2

Schematic of coordinate systems used to calculate the images formed from single molecule fluorescence. (a) Two equivalent parametrizations for expressing the orientation of a molecule’s transition dipole moment. Either a unit vector μ̂ or a pair of angles {Φ, Θ} is used. (b) Ray emanating from a molecule with trajectory defined by the unit vector r̂ having intensity Iff(ϕ,θ) ∝ sin2 (η), where η is the angle between μ̂ and r̂. The distribution of Iff(ϕ,θ) is thus a torus (pictured). (c) Approximations used for modeling defocus: If d ≪ r, then r′ ≈ r and θ′ ≈ θ. (d) Overview of the complete imaging system modeled by our simulations. Note that we assume θtube is small, and therefore the electric fields emerging from the tube lens will have a z-component that is nearly zero.

We now consider how an objective lens interacts with the electric field emanating from the dipole. The physical situation is diagrammed in Figure 2d. We specify the objective as having a focal length fobj as well as a maximum collection angle θmax. The objective acts in the following manner: Any ray emanating from the dipole with an inclination θ ≤ θmax will be rotated such that it is parallel to the optical axis. This transformation ensures that the S-polarized component of the electric field (with respect to the plane of the objective) remains unchanged, whereas the P-polarized component of the electric field will be rotated such that it is orthogonal to the optical axis. Unless acted upon by another optical component, the rotated ray will remain at a fixed distance away from the optical axis. Each of the rays collected by the objective will propagate parallel and in-phase with one another, until impinging upon the back focal plane located fobj behind the objective. We may now specify a new Green’s tensor, for determining the electric field at the back focal plane:The matrix Robj(ϕ,θ) is responsible for accomplishing the desired ray rotation and is expressed using the formulaIn eq 11, n0 is the refractive index at the back focal plane (normally n0 ≈ 1). The leading factor of n11/2/[n0 cos(θ)]1/2 was derived in ref (56) and ensures that the total energy contained in the hemispherical portion of the far-field collected by the objective is identical to the energy in the back focal plane. By plugging eqs 9 and 11 into eq 10, we arrive at an explicit expression for Gbfp:Note that in eq 12, r has been replaced by fobj. Also, the third row of Gbfp contains only zeros, ensuring that the electric fields will be rotated into the plane perpendicular to the optical axis. Furthermore, we recall that, if a ray leaves the dipole at a trajectory θ > θmax, then it will not be collected by the objective, and Gbfp will simply be the null-matrix. Although Gff was used to calculate the electric field at a point on a sphere, it is our intention to use Gbfp to determine the fields on a planar surface. It is therefore more natural to work in polar {ϕ, ρ} coordinates as opposed to spherical {ϕ, θ}. This transformation is accomplished using the following substitutions:In the transformation defined by eq 13, we have found it convenient to choose our units of length to be in terms of fobj. That is, ρ = 1 corresponds to a distance of fobj from the center of the back focal plane. Therefore, the radius of the circle within the back focal plane in which intensity is nonzero is ρmax = sin(θmax). Hence in polar coordinates, we obtainThe back focal plane electric field is simplyThe intensity at the back focal plane may also be calculated asNote that the defocus term, ei, no longer appears in eq 16. Hence, as long as the approximation d ≪ fobj is valid, the back focal plane intensity distribution will be independent of microscope defocus.

In Figure 3, we show the effects of molecular orientation upon the resulting Ibfp patterns. Note that changes in orientation will cause the intensity to shift to different regions within the back focal plane. When a molecule is oriented along the optical axis (Θ = 0°), the rays carrying the majority of intensity will propagate away from the dipole at an inclination θ > θmax, leading to a decrease in the objective’s overall collection efficiency. This feature is evidenced by the dimmer overall image. If an ensemble of molecules is fluorescing simultaneously, their back focal plane emission patterns will overlap. The back focal plane intensity distribution for an ensemble of randomly oriented molecules is also presented in Figure 3.

Figure 3

Representative back focal plane intensity distributions. Top row: orientation of dipole moment with respect to focal plane. Middle row: intensity distributions for a molecule in isotropic media. Bottom row: intensity distribution for a molecule at an air (n2 = 1) glass (n1 = 1.518) interface. Right column: intensity distribution resulting from an ensemble of randomly oriented molecules.

Though defocus alone does not perturb the back focal plane intensity distribution, inhomogeneities within the sample (such as a refractive index mismatch between a microscope coverslip and the medium in which the emitting molecule is embedded) will have a profound impact on Ibfp(ϕ,ρ). In the Appendix we provide formulas for properly augmenting Gbfp(ϕ,ρ) to account for the presence of a planar interface between the emitter and the objective, over which refractive index changes.[57,58] In Figure 3, the back focal plane intensity distributions modified to account for the presence of an air-glass interface are also presented. Note the bright ring on the outer edge of the back focal plane. This is caused by the enhancement of the evanescent electric field, and its conversion into propagating waves. Furthermore, one may note that when an interface is present, a molecule perpendicular to the interface exhibits superior collection efficiency to a molecule of parallel orientation.[59] The modified equation for bfp(ϕ,ρ) will be of use to us in section 4, when single molecules are simulated at an air–glass interface.

Calculation of the Image Plane Intensity for a Single Fluorescent Molecule: The Basis Function Approach

We now turn our attention to calculating the intensity distribution present in the image plane. When a phase mask is included, it is placed conjugate to the back focal plane (Figure 2d) using the 4f optical system. However, for simulation purposes, the additional relay optics need not be explicitly modeled. The phase mask may simply be rescaled by a factor of ftube/f4f (the magnification of the back focal plane by the first lens of the 4f system), then treated as if it were actually located inside the microscope. The physical effect of the phase mask is simply to multiply the back focal plane electric field Ebfp(ϕ,ρ,d) by a spatially varying phase-lag function ψ(ϕ,ρ). That isNote that the phase-lag function ψ(ρ,ϕ) is identical to the one discussed in the previous section. We have simply found it convenient to parameterize ψ in terms of polar as opposed to Cartesian coordinates. As shown in Figure 2d, conventional microscopes have a tube lens placed a distance ftube behind the back focal plane, which serves to focus the collimated rays exiting the objective into an image. It is feasible to derive an additional transformation matrix Rtube(ρ,ϕ) and then calculate the fields in the image plane by integrating the contributions from each individual ray.[55−57] However, there is a far simpler approach.[60] Referring to the diagram in Figure 2d, we realize that if a ray enters the objective at an inclination θ, it will leave the tube lens at the inclination θtube = sin–1[(fobj/ftube) sin(θ)]. However, for most microscopes, fobj ≪ ftube. For example, if θmax = 65°, fobj = 3 mm, and ftube = 180 mm, then the maximally inclined ray exiting the microscope will have the trajectory of only θmaxtube = 0.87°. Hence, we conclude that the paraxial theory of Fourier optics will be sufficient for our analysis. Practically, the small value of θmaxtube also ensures that electric fields exiting the tube lens will reside primarily in the plane orthogonal to the optical (z) axis. From the diagram, we note that the tube lens is placed one focal length behind the back focal plane and one focal length in front of the image plane. The electric fields between the two planes are therefore related by a scaled Fourier transform:Here, we have assumed that the medium surrounding both the image and back focal plane have refractive index n0. Note that we have expressed the Fourier integral in eq 18 using polar coordinates as opposed to Cartesian coordinates, used earlier in eq 1. As mentioned previously, the primed coordinates {ρ′, ϕ′} indicate position within the image plane, and the constant C incorporates a phase curvature term and an overall amplitude scaling factor. By substituting eqs 15 and 17 into eq 18, and moving the constant vector μ̂ outside of the integral, one can express Eimg aswhere we have definedIt is expedient to evaluate the integrals contained in eq 20 numerically, using the two-dimensional fast Fourier transform algorithm. We will denote the resulting components of the matrix Gimg using the following shorthand:For a given component, g, the superscript i refers to whether a component Gimg is contributing to either the x- or y-polarized portion of the resulting electric field Eimg, whereas the subscript j indicates the component of μ̂ by which g is multiplied. The image plane intensity distribution is thus calculated asThe first term enclosed in brackets above is the contribution to overall intensity from x-polarized light, whereas the second term is the y-polarized contribution. Furthermore, note that although Ibfp did not depend upon microscope defocus, Iimg is a function of d. If we expand the two terms in eq 22, we arrive at the following:where * denotes complex conjugation and indicates the real portion of the complex argument inside the brackets. Making the definitionssimplifies eq 23 to the following inner-product:The functions {XX, YY, ZZ, XY, XZ, YZ} may be regarded as basis functions defined over the image plane. Any measurement of Iimg by a camera will be the result of a linear superposition of these functions. Furthermore, their proper weighting may be straightforwardly determined by the emitting dipole’s orientation μ̂, and the amplitude A.

The basis function representation of Iimg has been used extensively by some authors.[61−63] However, it is our opinion that the computational advantage of this approach has been overlooked. Using a total of just six two-dimensional fast Fourier transforms, the proper basis functions may be calculated and stored for future use. Then, the proper intensity distribution for a molecule of arbitrary orientation may be simulated simply by computing the correct weighting factors. Additionally, one may incorporate the effects of a linear polarizer in a straightforward fashion. x/y-polarized images may be computed asIn general, a new set of basis functions must be calculated if one wishes to vary d, the amount of microscope defocus. Hence, when a simulation is designed, it is best practice to first decide upon the set of defocus values that are most relevant and then simulate a library of basis functions at different d, which are saved and later used to simulate single-molecule images. Figure 4 demonstrates a proof-of-concept simulation. A dipole fixed in orientation at {Φ, Θ} = {45°, 45°} is axially translated a distance up to d = 1000 nm from the microscope’s focal plane. We calculate Iimg at different values of d. The basis functions used to simulate Iimg at d = 1000 nm are also shown. For this simulation, we have assumed that no phase mask is included in the microscope, and hence ψ(ϕ,ρ) = 0 throughout the back focal plane (this configuration is termed clear aperture). Although we have found it convenient to specify our optical system using the parameters {fobj, ftube, θmax}, it is common practice to instead describe a microscope by its numerical aperture NA = n1 sin(θmax), and magnification M = (n1/n0)(ftube/fobj). Finally, it is worth noting that if one wishes to simulate the PSF of the optical system (the image of an isotropic point source), this may be accomplished simply by superimposing the images of three orthogonally oriented dipoles.[56]

Figure 4

Simulation of single-molecule images and basis functions. (a) Overview of simulation: A molecule (λ = 600 nm) is translated a varying distance d from the objective’s focal plane in isotropic media. We specify that the objective has an immersion medium of n1 = 1.518, and an NA of 1.4 (θmax = 67.26°). (b) Simulated images for a molecule with dipole moment oriented at: {Φ, Θ} = {45°, 45°}. (c) Basis functions used to simulate the defocused image d = 1000 nm. Note that the intensity color scale varies for each basis function. Units of length are specified in object space, i.e., before accounting for the magnification imparted by the objective/tube lens combination.

Single-Molecule Orientation Measurements With a Quadrated Pupil

Single-molecule microscopy features many methods for determining a fluorescent molecule’s dipole orientation. The first measurement techniques incorporated polarizing/analyzing optics in confocal microscope designs.[64] However, orientation measurements may be readily performed using widefield configurations as well.[31] By precisely fitting simulations to single-molecule image data, both the position and orientation of rotationally immobilized molecules have been simultaneously determined.[65] As a molecule is moved an increasing distance from the objective’s focal plane (Figure 4), the effects of orientation become more readily apparent upon the acquired image.[66] Thus, by simply defocusing a microscope, one may collect images that are more amenable to quantitative analysis.[67,68] However, as is evidenced in Figure 4b, the acquired image data will vary quite rapidly as a function of the precise amount of microscope defocus applied. Because neither defocus distance nor orientation is generally known beforehand, they must be simultaneously estimated[61] when data are fit to simulations.[61,69] Our recently developed phase mask for measuring orientation, termed the quadrated pupil,[53] addresses this problem by permitting orientation to be inferred without also requiring a precise depth estimate. Using a simple data analysis algorithm, and a customized dual-polarization 4f imaging system, we have achieved orientation measurement precisions of ∼2° for both Θ and Φ. In this section, we review the principles of this novel technique.

Figure 5 depicts our experimental apparatus. Using a polarizing beamsplitter, fluorescence exiting the microscope is separated into a reflected (R) and transmitted (T) channel, respectively containing S- and P-polarized light, as defined relative to the surface of the beamsplitter. Using the 4f optical processing configuration, the electric fields associated with the two polarization channels are Fourier transformed and projected onto an SLM using a pyramidal mirror (Figure 5a,c). The geometrical arrangement of our setup ensures that both the T and R channels will be polarized along the x-axis, defined relative to the SLM surface. This configuration is desirable because our liquid crystal SLM is capable of modulating only one polarization of incident light. After the SLM imparts a phase function ψ(x,y), another set of lenses performs a second Fourier transform and images the T and R emission channels onto separate regions of an electron multiplication charge coupled device (EMCCD) detector. The SLM is programmed with a pyramidal phase function (Figure 5b) consisting of four linear phase ramps:The constant C0 is set by the dynamic range of the SLM (∼6π), and C = C0/ρmax, where ρmax is the radius of the region in which intensity may be nonzero, as enforced by the numerical aperture, magnification, and the focal lengths of the lenses used in the 4f system. Intuitively, the function of this phase mask is as follows: Light falling into a given quadrant of the phase mask will be shunted into one of four separate points at the image plane. Because each polarization channel is independently phase modulated and imaged on a separate region of the EMCCD, fluorescence from a single molecule will appear as a total of eight separate “spots” on the detector. Because the distribution of intensity at the back focal plane will depend upon a given molecule’s orientation, the intensity distribution among each of the eight spots on the image sensor will also vary. (When isotropic emitters, such as fluorescent beads, are imaged, each of the image points will contain equal intensity.)

Figure 5

Dual-polarization/4f optical processing system. Adapted from ref (69) with permission. (a) Schematic diagram of experimental setup. EP and ES denote P- and S-polarized electric fields with respect to the beamsplitter, which subsequently are separated into ET and ER, the fields present in the transmitted and reflected polarization channels, respectively. (b) Plot of the phase function defining the quadrated pupil. Axis along which incident light is polarized is also sketched. (c) Geometry of our setup ensures that both the R and T channels are polarized along a single axis, so that the SLM can properly modulate all light emitted by the specimen.

A sample widefield fluorescence image is presented in Figure 6a showing both the R and T polarization channel images for single dye molecules, dicyanomethylenedihydrofuran-N-6 (DCDHF-N-6), spin-coated in a layer of 1% (by mass) poly(methyl methacrylate) (PMMA) dissolved in toluene, which served to immobilize them both in space and in orientation.[70] The molecules were excited with circularly polarized widefield illumination using a 514 nm laser at ∼1 kW intensity measured at the sample. Imaging was performed with a 1.4 NA objective (as modeled in our simulations). To quantitatively estimate orientation, a molecule of interest is identified in both the T and R channels, and the background-subtracted area-integrated f photons in each of the eight spots is calculated (Figure 6b). Photon counts are stored in a vector meas. The maximum-likelihood estimate of a given orientation is achieved by maximizing an objective function incorporating Poisson noise statistics:O(Θ,Φ) is related to the log-likelihood, l(meas,b|Θ,Φ), by addition of a constant which may be neglected because it does not influence the optimization procedure. b is the mean background fluorescence per pixel, and N is the number of pixels in the region used to calculate a given meas. The eight-element expected image vector sim is determined by simulating intensity-scaled polarized images of a single-molecule fixed at orientation {Θ, Φ} embedded at an air-glass interface (as described in the Appendix) and incorporating the quadrated phase mask using eq 17.

Figure 6

Representative data set adapted from ref (53) with permission. (a) Widefield image of single dye molecules. Both T and R channels are shown. Note that due to the geometry of the experimental setup, the R channel image undergoes an additional reflection before being projected onto the EMCCD. Inset: the pair of angles {Θ, Φ} denote a single point on the unit hemisphere. (b) Partitioning scheme used for processing measured and simulated data into the vectors meas and sim.

For our technique to produce accurate orientation estimates, the defocus distance between a given molecule and the objective’s focal plane need not be known with high precision. Microscope defocus does not dramatically alter the intensity distribution within the back focal plane and is therefore not a critical modeling consideration. To better understand this feature, we performed the following simulation: In Figure 7a, we simulate polarized images of a molecule embedded in isotropic media, with varying amounts of defocus applied (the quadrated pupil phase mask is in use). Throughout a range of |d| ≤ 150 nm, defocus will cause fine variations in the image recorded but will not drastically change the total amount of intensity contained in a given quadrant of a polarized image. If defocus exceeds this amount, the four spots in a given polarized image will either begin to overlap (d < −150 nm) or become too diffuse to extract a reliable intensity estimate (d > 150 nm). To further quantify this effect, the intensity-normalized components of the vector sim are plotted over a 0.5 μm range (Figure 7b). As expected, the components of this vector do not change appreciably over the range |d| ≤ 150 nm. This implies that even though the images that are acquired at different defocus settings will be slightly altered, the data input into our estimation algorithm will be nearly identical. Though knowledge of defocus need not be exact, it is necessary to have accurate information about any refractive index variations throughout the sample. Changes in refractive index will affect the back focal plane intensity distribution (Figure 3) and must therefore be well accounted for in simulation. So long as simulations accurately model any refractive index mismatches, this technique suffers no loss in accuracy. However, when samples such as cells are imaged, it is important to ensure that inhomogeneity within the sample is not so severe as to significantly alter the back focal plane intensity such that it is no longer well-modeled by simulations—such verification may be carried out by directly inspecting the back focal plane using a Bertrand lens.

Figure 7

Invariance to defocus. (a) Representative simulated images of a single molecule {Θ = 40°, Φ = 25°}. (b) Plot of normalized entries of sim as a function of defocus d. In the ±150 nm range indicated, the components of sim change minimally. For this simulation, an isotropic medium was assumed (no index mismatch).

As a proof-of-concept, maximum likelihood orientation measurements for two representative DCDHF-N-6 molecules are shown in Figure 8a,b. To benchmark our precision, 20 successive 1 s frames of data were acquired, and the orientation of the same molecules was repeatedly estimated. The estimated angles {Θ, Φ}, are plotted as points on a unit hemisphere (inset in Figure 6a). Furthermore, the objective function O(Θ,Φ) may be evaluated throughout the unit hemisphere, to gauge the relative likelihood of different orientations. In Figure 8c,d, we demonstrate our technique to be insensitive to minor defocus errors. The objective lens of our microscope was translated in 50 nm steps, with 11 frames of data recorded at each step. When the focal plane is within ±150 nm of the layer of single molecules, the orientation measurements are largely invariant.

Figure 8

Measurement results adapted from ref (53) with permission. (a) and (b) Orientation measurements for two molecules. At left: raw data and simulated images obtained from the mean orientation estimate. Center: repeated orientation measurements for the same molecule, plotted on the unit hemisphere. The 2σ ellipse computed from the data-covariance matrix is plotted in green. Right: Magnified view of the region of interest. For the molecule in (a), the mean orientation was: {Θ = 42.2, Φ = 242.2} with a standard deviation of {σΘ = 1.8°, σΦ = 1.7°}. An average of 2370 photons were detected per exposure. For the molecule in (b), we found {Θavg = 73.9°, Φavg = 326.9°} and {σΘ = 5.8°, σΦ = 4.3°}. An average of 921 photons were detected. (c) Orientation measurements for a single molecule over a ±150 nm range. Standard deviations at each depth are depicted by blue bars. (d) Sample images taken at different focal planes demonstrate robustness to defocus. For this molecule, an average of 916 photons per exposure were detected.

Bisected pupil for studying single-molecule orientational dynamics and its application to 3D super-resolution microscopy

In this section we present an adaptation to the quadrated pupil phase mask design, termed the “bisected pupil”, which enables the lateral (x/y) position of fluorescing emitters to be estimated,in addition to emitter depth—thus achieving three-dimensional super-resolution imaging. Implementation details of our depth estimation procedure are discussed. Although the quadrated pupil phase mask design provides accurate, high precision orientation estimates for all actively fluorescing molecules in a field of view, it is not a particularly well-suited phase mask for the task of localizing individual molecules (even in x and y) for two reasons: When the quadrated pupil is used, emission from a given molecule is divided into eight “lobes” in two polarization channels. In the presence of even modest background, distributing photons over such a diffuse region will have negative consequences from a signal-to-noise standpoint.[16] Furthermore, in the context of super-resolution imaging, it is necessary to detect large quantities of individual molecules over many frames of data. It is difficult for automated detection algorithms to properly identify molecules in the presence of modest background fluorescence, if signal photons are spread over many pixels. To confidently identify molecules from raw image data, and obtain both precise position and orientation measurements, we propose a compromise—instead of partitioning the back focal plane using four phase ramps, we instead use only two. The resulting phase mask, the bisected pupil,[71] splits emission into only four lobes over two polarization channels, and may be much more readily applied to single-molecule localization analysis.

Super-Resolution Imaging in 3D with a Bisected Pupil

The functional form of the bisected pupil phase mask can be expressed asAnalogous to eq 27, C0 and C are tunable constants that control the slope and magnitude of phase variation throughout the phase mask. Figure 9a depicts the bisected pupil phase mask design. Figure 9b shows a series of simulations of an isotropic emitter (i.e., three orthogonal dipoles superimposed) imaged in the T- and R-polarization channels at different defocus depths. Due to the geometry of the setup, the lobes in the T-channel image are rotated 90° from those in the R-channel image. Furthermore, we make the following key observation: as the depth of the emitter varies, the two lobes in a given polarization channel will appear to contract (d < 0) or expand (d > 0). This feature suggests that by measuring the interlobe distance for the image of a given emitter, the precise z-position of the object may be inferred. In Figure 9c, we translate an objective lens relative to a fluorescent bead spin-coated onto a microscope coverslip. By fitting Gaussian functions to the two lobes in a given polarization channel, we are able to create a lookup-table relating lobe-spacing to depth. Furthermore, by calculating the midpoint between the centers of the two Gaussians, the lateral position may be determined.

Figure 9

Overview of the bisected pupil. Adapted from ref (71) with permission. (a) The bisected pupil phase mask is plotted, and the polarization axis of incident light is indicated. (b) Simulations of an isotropic emitter imaged with the bisected pupil, at varying depths (color scale has been renormalized for each d, to display fine features of the PSF). (c) Calibration curve relating lobe separation distance to depth. This calibration was acquired by translating the objective lens relative to a fluorescent bead. A polynomial curve fitted to the data is also shown, indicating a nearly linear relationship.

Given a recipe for determining the positions of single molecules in three dimensions, the technique of super-localization may be leveraged to construct super-resolved widefield images of extended biological structures. A variety of methods, such as (f)PALM and STORM,[23−25] have achieved resolution enhancements an order of magnitude below the diffraction limit using the following procedure: (1) By optical or chemical means, the concentration of actively fluorescing molecules labeling a structure of interest is limited such that their individual emission patterns become distinguishable in the image plane. (2) Single molecules are then super-localized to within a few tens of nanometers by fitting a model function (two Gaussians in the case of the bisected pupil) to the image recorded on a detector. (3) Multiple frames of data are recorded, and all detected molecules are individually localized, such that the labeled structure is fully sampled. (4) The underlying structure is then reconstructed by plotting the positions of all localized molecules. We will refer to super-resolution methods employing this strategy as “single-molecule active control microscopy” (SMACM). To demonstrate SMACM imaging in three dimensions, we used the dual-polarization 4f system to image microtubules in fixed BSC-1 cells, immunolabeled with the dye Alexa Fluor 647 (Invitrogen). The sample was imaged in a buffer containing β-mercaptoethylamine thiol and a glucose, glucose oxidase, and catalase oxygen-scavenging system.[72] By imaging at intensities ∼10 kW measured at the sample, with a circularly polarized 641 nm laser, the individual dye molecules were forced to “blink” on and off, permitting their individual emission patterns to become visible. See Figure 10a for sample frame of raw data. After a 10 min sequence of 30 ms exposures, individual molecules were identified in the T-polarization channel, super-localized, and binned into 25 nm pixels, which were color coded according to depth. The resulting image is shown in Figure 10b. In comparison to a conventional diffraction-limited widefield fluorescence image (inset), the resolution enhancement is clearly evident. For this data set, an average of 2000 photons per polarization channel per molecule was detected. The background was on average 20 photons per pixel. This signal and background level permitted molecules to be localized with ∼20 nm precision.

Figure 10

Super-resolution imaging in 3D with a bisected pupil. Adapted from ref (71) with permission. (a) Frame of raw image data showing blinking Alexa-647 molecules. Both T and R channels shown. (b) Super-resolution image of microtubules in fixed BSC-1 cells.

Bisected Pupil Imaging Reveals Pronounced Rotational Mobility

In the previous subsection, our analysis ignored the dipolar features of single-molecule images, discussed at great length in sections 4 and 5. Is this omission justifiable? Previous studies have suggested that fitting simplistic model functions that do not properly account for dipole emission to single-molecule images can cause systematic localization errors.[73] This effect is accentuated by slight microscope defocus (|d| ≤ 250 nm), which can induce mislocalizations on the order of 200 nm.[74] These huge localization errors are most prominent when asymmetric features arise in the acquired data, due to molecules with transition dipole moments tilted away from both the optical axis and the plane of the microscope coverslip (Θ ∼ 45°). A number of studies have sought to mitigate these errors[61,75] and benchmark the effects of orientation upon localization precision limits.[76] In previous work, we demonstrated that the three-dimensional positions of molecules immobilized in a polymer could be accurately inferred by first estimating their dipole orientation and subtracting the respective systematic localization error using a lookup-table.[69] However, such effort may not always be necessary. In biological specimens, molecules labeling structures often undergo some degree of rotational motion, depending upon the specific probe, and the labeling method employed. As a molecule’s rotational mobility increases, its fluorescence image will appear as that of a superposition of immobilized dipoles. The molecule will thus resemble an isotropic emitter, mitigating any localization errors introduced by orientation. To characterize rotational mobility, it is often assumed that a molecule is free to rotate about a fixed axis, within a cone defined by an angle α (Figure 11a).[77] This model may be augmented with rotational diffusion and excited state fluorescence lifetime data, making it possible to estimate the amount of rotational freedom necessary to mitigate localization error. Our calculations indicate that if α ≥ 65°, lateral localization errors are bounded to fewer than 10 nm.[78]

Figure 11

Rotational mobility measurements with a bisected pupil. Adapted from ref (71) with permission. (a) The “rotation within a cone” model: A molecule is assumed to have a mean orientation described by the pair of angles {Θ, Φ}, and may rotate to any orientation within the cone specified by the angle α. (b) Diagram indicating the regions of an image that are summed when the linear dichroism (LD) and lobe asymmetry (LA) of a molecule are calculated. (c) Histograms of simulated single molecule images (blue) indicate that rotational mobility is high. Experimentally acquired data (red) most closely matches simulation using α = 75°. Note that as rotational mobility increases, standard deviations σLA and σLD, of histogrammed data decrease. (d) Super-resolution images color coded according to LD and T-channel LA.

To gauge the rotational freedom of the Alexa-647 molecules used in our super-resolution imaging experiment, we define two experimentally measurable quantities from the bisected mask images: The linear dichroism (LD)[79] and lobe asymmetry (LA).[69,80] These quantities are defined as (Figure 11b):where AT,R is the number of background-subtracted photons contained in one polarization channel attributed to a given molecule and L1,2 is the number of photons contained in one lobe of the single-molecule image in a given polarization channel (different lobe asymmetries may be calculated for the T- and R-polarization channels). In general, these quantities will vary as a function of a molecule’s mean orientation. However, overall, both of these parameters tend to decrease in magnitude as the rotational freedom, α, increases. To quantitatively investigate this feature, an ensemble of 10 000 single-molecule images was simulated using the bisected pupil phase mask. The resulting LD and (T-channel) LAT values were then histogrammed. The mean orientation of each molecule was drawn randomly from a uniform distribution, and α was kept fixed. This simulation was repeated three times using α = {25°, 50°, 75°} (Figure 11c). EMCCD noise statistics were simulated,[76] assuming a mean of 2000 signal photons per polarization channel for a molecule oriented parallel to the optical axis, and 20 photons per pixel of background. For these simulations, we adjusted detected signal photons to account for the relative pumping/collection efficiencies of molecules at different mean orientations with respect to the optical axis. The molecules were assumed to be immersed in water, in perfect focus, yet far enough above the water–glass interface such that the evanescent field decayed almost completely before it could be converted into supercritical light. The numerical aperture of our objective was thus effectively limited to 1.33. Furthermore, molecules were assumed to visit all orientations within the cone described by α with equal frequency. However, more sophisticated rotational diffusion models may be employed.[78] By comparing our simulated LD and LAT values to a histogram constructed from 10 000 single-molecule images drawn from our experimental data set, we find that our experimental data most closely matches the α = 75° trial (Figure 11c). It is therefore likely that the majority of emitters are almost completely rotationally mobile. Furthermore, by color-coding pixels in a super-resolved image according to LD and LAT, we find that these parameters are uniformly low throughout a field-of-view. We thus conclude that orientation-induced mislocalizations have not degraded or distorted this image. Scrutinizing the data in Figure 11c, we note that the experimentally obtained histograms have longer “tails” than the simulations. It is thus entirely possible that a small, yet detectable, subpopulation of immobile molecules may, in fact, be present in this sample. Such matters will be an excellent direction for future research.

Conclusion

In this article, we have summarized the usage of Fourier processing with a variety of phase masks and imaging systems for tracking fluorescent probes, imaging biological structures, and determining the orientation and rotational mobility of single molecules. Our lab’s application of this novel set of experimental techniques is ongoing. For example, in the previous section our analysis of LA and LD statistics confirmed that orientation caused minimal degradation of localization accuracy for a specific sample. However, we discourage making broad assumptions from this particular study. For example, researchers have reported significant polarization anisotropies in actin specimens.[81,82] Because much additional information normally hidden in the pupil plane of the microscope is transferred into the final image on the camera, optical Fourier processing will thus serve as a useful diagnostic and measurement tool for future single-molecule imaging applications.