Precise Three-Dimensional Scan-Free Multiple-Particle Tracking over Large Axial Ranges with Tetrapod Point Spread Functions.

We employ a novel framework for information-optimal microscopy to design a family of point spread functions (PSFs), the Tetrapod PSFs, which enable high-precision localization of nanoscale emitters in three dimensions over customizable axial (z) ranges of up to 20 μm with a high numerical aperture objective lens. To illustrate, we perform flow profiling in a microfluidic channel and show scan-free tracking of single quantum-dot-labeled phospholipid molecules on the surface of living, thick mammalian cells.

Single-particle tracking (SPT), in which the trajectory of a moving individual molecular label, quantum dot, or nanoparticle is determined from a series of images, provides a valuable tool for a wide range of biological applications. Information inferred from the extracted particle trajectory can shed light on physical properties such as particle size, conformation, and the local environment,[1−6] because observing the motion of single particles directly unmasks nanoscale behavior such as diffusion,[7] directed motion,[8] or anisotropy;[9] methods to enhance SPT would be valuable to many fields of study.

SPT techniques are typically based on frame-by-frame localization of the particle. Namely, a series of time-sequential images (frames) is captured using a microscope, and each frame is analyzed to yield the current position of the particle. In some applications, the extracted positions are in two dimensions (2D), comprising lateral, or x,y coordinates. The noisy and pixelated 2D detector image of the particle is analyzed, for example, by centroid or Gaussian fitting[10] to yield the estimated x,y coordinates of the particle. However, because most samples of interest are inherently three-dimensional (3D), the full physical behavior of the tracked object is in many cases only revealed by analyzing its 3D trajectory.[1,11] The 3D trajectory of a moving particle can be extracted in several ways. For example, a particle can be followed by using a feedback control loop based on moving a 3D piezo stage according to the reading of several detectors (e.g., photodiodes).[12,13] While providing a very precise temporal and spatial trajectory this method is inherently limited to tracking a single particle.

Alternatively, scanning methods, such as confocal microscopy,[14] can be implemented in which an illumination beam or the focal point of the microscope (or both) are scanned over time in three dimensions to yield a 3D image of the object. Naturally, any scanning method is limited in its temporal resolution, because at a given time only a small region is being imaged. In order to realize fast, simultaneous tracking of several particles in 3D, a scan-free widefield approach is required.

To efficiently encode information regarding the axial dimension (z), an optical system must be modified from that of a standard microscope. One possible modification is the “multi-plane” approach, based on simultaneous imaging of two or more planes in object space.[15−17] This method requires simultaneous acquisition of multiple images, and is currently applicable to an axial range of ∼4 μm.[17]

Importantly, it is possible to extract 3D position information from a single widefield 2D image, by modifying the microscope’s point spread function (PSF), namely, the image that is detected when observing a point source. Examples of PSF alterations that have been used for 3D tracking and imaging under biological conditions include astigmatism,[18−20] the double-helix PSF,[21−23] the corkscrew PSF,[24] the bisected-pupil PSF,[25] and an Airy-beam-based PSF[26] with applicable z-ranges of ∼1–2 μm for astigmatism and the bisected pupil PSF and ∼3 μm for the double-helix, corkscrew, and Airy PSFs.

Recently, we developed a method to design information-optimal PSFs for 3D imaging[27] based on numerically maximizing the information content of the PSF. The resulting “Saddle-Point” PSF exhibits superior 3D localization precision over existing PSFs. However, despite gradual improvements in PSF designs over recent years in terms of achievable precision, they are still quite limited in terms of their applicable z-range. Currently, the z-range of existing PSF designs is limited to ∼3 μm, posing a major limitation for applications requiring “deep” imaging. For example, the thickness of a mammalian cell can often be larger than 6 μm[28] and in the case of cells grown on cell feeder layers or in 3D cell cultures, which are becoming increasingly popular in the biological community, samples are obviously much thicker.[29] This means that, for example, tracking the trajectory of a labeled protein over this axial range with high precision by using any existing PSF design cannot be accomplished without scanning or multi-plane imaging.

Here, by utilizing the information maximization framework,[27] we present PSFs that enable precise 3D localization over a depth range far larger than the applicable depth ranges of existing designs. By setting the optimization parameters to correspond to the desired depth range, we engineer specific PSFs yielding precise 3D localization optimized over that range. The resulting PSFs belong to a family we dub the “Tetrapod” PSFs due to the tetrahedral shape they outline in 3D as a function of the emitter’s axial position. We demonstrate the utility of these exceptionally large-range PSFs for two experimental applications of wide interest. First, we optimize a Tetrapod PSF for a 20 μm z-range and use it for flow-profiling in a microfluidic channel. We then use a Tetrapod PSF optimized for a 6 μm z-range under biological conditions; namely, we track single quantum-dot labeled lipid molecules diffusing in live mammalian cell membranes.

The imaging system is a standard (inverted) microscope, augmented by a 4f optical processing system,[30] as shown in Figure 1. The PSF of the microscope is modified from that of a standard microscope by controlling the phase of the electromagnetic field in the Fourier plane of the 4f system. This can be done by placing a dielectric phase mask[31] or a liquid crystal-based spatial light modulator (SLM)[6] in the Fourier plane. For the experimental implementation here, use an SLM.[6]

Figure 1

Experimental implementation of a 4f optical processing system. The PSF is modified by a phase mask (or spatial light modulator) placed in the Fourier plane.

With this imaging setup, the image formulation model is given by[32]where I(u, v) is the image, or the intensity in the camera plane, of a point source located at position (x, y, z) in sample space, relative to the focal plane and the optical axis (z). The field in the pupil plane, caused by the point source, is denoted by E(x′, y′), and represents the 2D spatial Fourier transform with appropriate coordinate scaling. The complex function P(x′, y′) is the pattern imposed in the pupil plane by a phase mask or an SLM (see the Supporting Information for mathematical details).

Designing a PSF that enables precise 3D localization over a large z-range under high-background conditions (due to out-of-focus fluorescence or sample autofluorescence) is a challenging task. Consider the demands for such a PSF: On one hand, the microscope optics must concentrate the light into a relatively small region throughout the applicable z-range in order to overcome background noise. On the other hand, the PSF must contain high Fisher information, that is, features that change sufficiently quickly as a function of z, such that its shape encodes the z position of the emitter with high “recognizability” or “z-distinctness”. This translates into high statistical localization precision when localizing under noisy conditions.

Fortunately, the theoretical precision of a given PSF can be quantified by the Cramer Rao Lower Bound (CRLB).[15,33−35] The CRLB is a mathematical quantity indicative of the sensitivity of a measurement to its underlying parameters.[36] More specifically, the CRLB corresponds to the lowest possible variance in estimating these parameters with an unbiased estimator. In our case, the measurement is a noisy, pixelated manifestation of the PSF (the 2D image), and the underlying parameters are the 3D coordinates of the emitter, as well as its brightness expressed as total signal photons, and a background level.

Therefore, the challenge of designing an optimal PSF can be treated as an optimization problem with the objective function being the CRLB, as we have recently demonstrated.[27] Indeed, our approach to PSF design is purely algorithmic and works as follows: Given the system parameters, such as magnification, numerical aperture, background and signal levels, and a (Poisson) noise model, we build a numerical imaging model based on eq 1. We then use this model to solve the optimization problem of finding the Fourier phase pattern P(x′, y′) that would yield the PSF with the lowest theoretical localization variance (equivalently, the lowest CRLB). The CRLB is the inverse of the Fisher information matrix.[36] Practically, therefore, the objective function being minimized is the mean trace of the inverse of the Fisher information matrix (corresponding to mean x,y,z CRLB) over a finite set of N unique z-positions in a defined z-range. That is, we wish to solve the following minimization problem:where in eq 2, F is the 3-by-3 Fisher information matrix associated with the x–y–z localization precision associated with the PSF at the jth z-position. This optimization can be performed over a subset of functions, for example, Zernike modes.[37] See the Supporting Information and ref (27) for further details.

Running our optimization routine with different specified z-ranges yields different phase masks (and corresponding PSFs). However, the resulting PSFs share very strong common characteristics. Namely, for any tested z-range (from 2 to 20 μm) these PSFs consist broadly of two distinct lobes with growing transverse distance between them as the emitter departs from the microscope’s focal plane. The orientation of the two lobes of the PSF is rotated by 90° above and below the focal plane. This family of PSFs is therefore dubbed the Tetrapod PSFs due to the 3D tetrahedral shape they trace out as the point source is moved in z (the axial direction).

Two example Tetrapod masks, along with the corresponding PSFs (calculated and experimentally measured), optimized for a 6 μm range and a 20 μm range, are shown in Figure 2. These masks were optimized to work in a high background scenario, corresponding to live-cell imaging conditions, namely, 3500 signal photons and a mean background of 50 photons per pixel. We note that “signal photons” refers to photons originating from the emitter and detected by the camera, integrated over the PSF. The PSF measurements are obtained by imaging a 200 nm fluorescent bead attached to a microscope coverslip and scanning the microscope objective such that the focal plane is above or below the bead. Note that the physically “sensible” requirements mentioned above, namely, concentrating the light into lobes and having the PSF shape vary quickly as a function of z, are indeed fulfilled by the resulting PSF, however this was achieved without adding these requirements explicitly; these beneficial features arise naturally as a consequence of optimizing our objective function based on the CRLB.

Figure 2

Tetrapod masks, optimized for z-ranges of 6 μm (a–d) and 20 μm (e–h). Tetrapod phase patterns designed for 6 μm and 20 μm (a,e). Numerical PSF calculation for various z-positions (b,f) and experimentally measured bead images (c,g), each image normalized by maximum intensity. (d,h) Numerically calculated precision, defined as (CRLB)1/2 for x, y, and z determination, using 3500 signal photons on a background of 50 mean photons per pixel.

The calculated precision (standard deviation, defined as (CRLB)1/2) for a signal of 3500 photons over a mean background of 50 photons per pixel is also plotted (Figure 2d,h). According to the CRLB calculations, under these signal-to-noise conditions the PSF is capable of exhibiting a mean precision of 12 nm, 12 nm, 21 nm (29 nm, 29 nm, 53 nm) in estimating x, y, and z, respectively, using the 6 μm PSF (20 μm PSF). See the Supporting Information for further details on the performance of Tetrapod PSFs optimized for different z-ranges.

In order to experimentally demonstrate the applicability of the Tetrapod PSFs, we first use the 20 μm Tetrapod mask for experimental flow profiling in a microfluidic channel (Figure 3). Microfluidic devices are useful for obtaining various measurements of interest, ranging from molecular diffusion coefficients[38,39] or pH,[40] spanning 3D vascular modeling[41] to inexpensive clinical diagnostic applications.[42] The use of PSF engineering provides a simple (scan-free) and precise method for 3D flow profiling in such devices.

Figure 3

Microfluidic channel setup. Water with fluorescent beads (200 nm diameter, 625 nm absorption/645 nm emission) is flowing through a microchannel, placed on top of a microscope objective of an inverted microscope. As the beads flow, they are excited by a laser (641 nm), and their fluorescence signal is captured. Two beads are illustrated.

Initially, we investigate the well-studied laminar flow regime.[43] Water with a low concentration (∼0.5 pM) of fluorescent beads is flowing through a glass microfluidic channel with a semicircular cross-section (50 ± 8 μm (width) × 20 ± 3 μm (height) near the center of the channel). A 641 nm laser illuminates the sample, the widefield fluorescence signal from the flowing beads is recorded, and a video is taken (5 ms exposures at 20 Hz). The beads are localized as they flow, and the profile of the flow is then obtained by analyzing their trajectories, a technique known as particle-image-velocimetry (PIV[44,45]). Three-dimensional localization of each bead in each frame is achieved using maximum-likelihood estimation[15] based on fitting each image to a numerical model of the PSF and taking into account objective defocus and refractive index mismatch between sample and mounting medium (see Supporting Information for details).

Figure 4a shows an example raw-data frame (see also Supporting Information Movie 1) where three beads at different x,y,z positions can be simultaneously seen. By accumulating many such frames (∼16 000), the mean flow velocity as a function of x, y, and z (namely v, v, v) is calculated. Figure 4b shows the y–z profile of the flow (which is in the x-direction), whereas Figure 4c,d shows 1D cross sections near the center of the channel. The v profile (black) is seen to be quite reasonably parabolic, while the mean v and v (blue, red) are very small in comparison (mean values of v, v are 0.36, −0.39 μm/s, respectively, 2 orders of magnitude smaller than the maximal v). This fits well with a laminar flow model, assuming no slip conditions,[43] which is applicable for our experimental conditions, where the Reynolds number is Re ≈ 4 × 10–4.

Figure 4

Laminar flow measurement. (a) Example raw frame, showing three emitters at different x, y, z positions, flowing in the x-direction. (b) Experimentally derived two-dimensional mean v map, averaged over x (y–z cross-section). (c,d) One-dimensional slices from (b), showing mean v, v, and v velocities. As predicted by a laminar flow model, v (black) has a parabolic profile, whereas v and v (blue, red) are negligible by comparison. Error bars represent ±1 s.d.

Various quantities of interest can be obtained by a quantitative study of the measured bead trajectories. First, by analyzing mean-squared-displacement (MSD) curves in the y- and z-directions (i.e., orthogonal to the flow), we infer a mean diffusion coefficient of 1.20 ± 0.13 (1.24 ± 0.19) μm2/sec in the y (z) direction. This compares very well with the theoretical value given by the Einstein–Smoluchowski relation for a 200 nm spherical diffuser in water[46,47] of 1.08 ± 0.03 μm2/sec. Second, from the MSD curve intercepts the localization precision can be approximated.[48] The resulting derived precisions are 76 nm (87 nm) in the y (z) estimation. See Supporting Information for MSD curves and further details. A total number of 3773 emitters were tracked, producing 52 332 data-points (localizations).

To demonstrate a full 3D velocity profile, we image the microfluidic channel right under the input facet where the bead solution enters the microfluidic channel (see inset cartoon in Figure 5). The beads are then imaged as they enter the channel, thereby exhibiting considerable flow also in the z-direction. Figure 5 shows the resulting flow profile. Figure 5a contains 3D trajectories of 100 beads with color coding normalized per trajectory start (blue = first frame in trajectory) to end (yellow = last frame in trajectory). A typical trajectory lasts ∼1.5 s. Figure 5b shows an x–z cross-section of the flow, near the center of the channel (in y). See Supporting Information Movies 2 and 3 for further visualizations.

Figure 5

Three-dimensional flow measurement. (a) Three-dimensional flow trajectories of a 100-bead subset at the entrance facet of the microfluidic channel (see inset in (b)), color-coded by normalized trajectory duration (blue, start; yellow, end). Typical trajectory duration ∼1.5 s. (b) An x–z slice. The flow is profiled over ∼30 μm in z. The data is binned in 3 × 3 × 3 μm3x–y–z bins, arrow length linearly encodes velocity (longest arrow corresponds to 22.5 μm/s).

There are several factors contributing to localization error in the described flow experiments. One factor is signal-to-noise ratio, determined by the finite number of signal photons relative to background photons. However, the measured beads are very bright (number of signal photons per frame on the order of ∼100 000), and the background is sufficiently low (a few photons per pixel) such that this is not a major contributor. Motion blur is another cause for localization error, however this as well is probably not a major factor because our exposure time (5 ms) is short in light of the velocities and diffusion rates of the measured beads, and our simulations indicate that this has minor effects on localization precision.

The most dominant contribution to localization error comes from model mismatch, namely, the model to which each measured PSF is fit deviates from the real PSF. This is partly because of aberrations in the optical system, and mostly because of aberrations related to refractive index mismatch. As is well-known,[30,49,50] the PSF of a point source (bead) in water is somewhat different from the PSF of a bead on a coverslip, and therefore difficult to calibrate experimentally because it is challenging to locate point sources at precisely known depths. Our imaging model does include the effect of refractive index mismatch, however even this modeling has limited precision. The problem of correcting for index-mismatch-related aberrations in microscopy is a long-standing one, and a subject of ongoing research. The use of more sophisticated numerical models and possibly calibration methods (for example, see ref (51)) will decrease any localization error that accompanies these kinds of measurements.

Another contribution to localization error comes from overlapping PSFs of closely spaced emitters. When localizing an emitter, photons from a different nearby emitter can contribute to nonhomogenous background noise and degrade localization performance. For a qualitative discussion and an illustrative example, see Supporting Information.

Naturally, an appealing application of PSF optimization is the study of biological phenomena by 3D tracking of nanoscale objects. In order to demonstrate the utility of the Tetrapod PSF under real biological conditions, we use a Tetrapod mask optimized for a 6 μm z-range to track the diffusive motion of single quantum dots attached to lipid molecules in live-cell membranes. Figure 6a shows an example frame from a tracking experiment following the motion of a lipid-anchored quantum-dot (phosphoethanolamine (PE)) on the surface of a living HeLa cell (see Supporting Information). The extracted 3D trajectory is plotted in Figure 6b. The mean number of detected signal photons per 50 ms frame is ∼10 000 with a mean background of ∼40 photons per pixel. The precision in this measurement is estimated to be 10 nm in the x–y coordinates and 17 nm in the z-coordinate. This is measured by localizing immobilized quantum dots on the surface of the sample’s coverslip and averaging the standard deviation in localization for several defocus values (see Supporting Information).

Figure 6

Long-term 3D tracking of a Qdot-labeled PE lipid on the surface of a live HeLa cell. (a) Brightfield impression with overlaid fluorescence channel (one 50 ms frame), showing signal from a quantum-dot-labeled PE lipid. Scale bar: 10 μm. (b) Inferred 3D trajectory as a function of time (55 s total), color-coded with time progression, and planar projections of the motion shown in gray on each of the bounding surfaces. The total motion over an axial range of 5.67 μm is mapped. Maximum-likelihood estimation on a frame-by-frame basis produces the trajectory. Measured fluorescence data is given in Supporting Information Movie 5.

Interestingly, the 3D trajectory of the quantum dot tracked in Figure 6 seems to be constrained to an approximately spherical surface. The sphere that the trajectory outlines is in fact visible in the white light image (Figure 6a) and is probably a detached bleb from a nearby cell, pressed against the cell membrane from the outside. When fitting the trajectory to a sphere, the radius corresponds very well with the value obtained from the white light image (see Supporting Information Movie 4).

While the example in Figure 6 shows a single tracked molecule, a major advantage of the PSF-engineering based tracking method is that it allows for simultaneous tracking of multiple emitters. See the Supporting Information for additional cell tracking examples, containing multiple molecules tracked simultaneously.

Importantly, these PSFs are directly applicable to single-molecule localization microscopy.[52−55] In order to demonstrate such capabilities, we first immobilize single fluorescent dye molecules (Alexa Fluor 647) on a coverslip. We then excite the molecules and measure their fluorescence using a 6 μm Tetrapod PSF. Each molecule’s position is then localized repeatedly. We repeat this for various defocus values throughout a ∼7 μm z-range. For a mean number of ∼6000 detected signal photons and ∼38 background photons per pixel, the mean statistical localization precision, namely the standard deviation of localizations, averaged over the entire z-range, is 15, 12, and 29 nm in x, y, and z, respectively. See the Supporting Information for this result.

In summary, we present an imaging modality based on optimized Tetrapod PSFs, capable of high-precision imaging throughout an unprecedented and tunable axial range. We experimentally demonstrate large-axial-range tracking in a microfluidic device, tracking under biological conditions of Qdot-labeled molecules diffusing on the membrane surface of live mammalian cells, as well as single-fluorophore localization capabilities over a ∼7 μm axial range. Use of the Tetrapod PSFs provides a solution to a long-standing problem of high-precision, scan-free tracking of multiple emitters over an exceptionally large z-range. Note that the experiments described in this work were performed using a liquid-crystal-based SLM in the Fourier plane. However, replacing the SLM with a specially fabricated phase mask or a deformable mirror will substantially increase photon efficiency, which is a limiting factor in precision for single-molecule imaging. Further, any localization shifts arising from fixed emission dipole orientations apply equally well to this PSF as to any other, and these may be addressed in various ways.[23,56,57]