Ultrahigh accuracy imaging modality for super-localization microscopy.

Super-localization microscopy encompasses techniques that depend on the accurate localization of individual molecules from generally low-light images. The obtainable localization accuracies, however, are ultimately limited by the image detector's pixelation and noise. We present the ultrahigh accuracy imaging modality (UAIM), which allows users to obtain accuracies approaching the accuracy that is achievable only in the absence of detector pixelation and noise, and which we found can experimentally provide a >200% accuracy improvement over conventional low-light imaging.

Super-localization microscopy[1-6] comprises an ever-expanding set of techniques that rely on the pinpointing of the locations of individual fluorescent molecules for purposes such as the high resolution reconstruction of subcellular structures and the high accuracy tracking of the movement of proteins inside cells. Common to these techniques is the use of a pixelated detector to record the fluorescence signal collected by the microscope, yielding images from which the molecules are subsequently localized. The detector is typically a charge-coupled device (CCD) detector or an electron-multiplying charge-coupled device (EMCCD) detector[7]. Both types of detectors accumulate photoelectrons in their pixels in proportion to the number of detected photons, and produce a digitized image via a readout process. The EMCCD detector, however, has a multiplication register that amplifies the number of photoelectrons before they are read out, with the intended purpose of augmenting weak signals above the readout noise floor.

A key impediment to the localization of a fluorescent molecule with ultrahigh accuracies is the fact that CCD and EMCCD detectors deteriorate the acquired image in two major ways. One, they pixelate the image, thereby substantially lowering its resolution. To improve the localization accuracy in the context of scanning microscopy, for example, an approach[8] has been described that addresses the issue of pixelation via the detection of single photons. Two, they introduce noise to the image. For the CCD detector, the primary noise source is the aforementioned readout noise, which overwhelms weak signals and renders the CCD detector unsuitable for extreme low light imaging. For the EMCCD detector, besides the readout noise, the signal amplification is itself an important noise source due to its stochastic nature. Pixelation and noise can lead to substantially lower accuracies with which the positions of molecules can be estimated, compared to what would be possible if the image were recorded with an ideal detector that preserves the image exactly as produced by the microscope.

The deteriorative effects of detector pixelation and noise become especially consequential under low light conditions, since even without these effects, the accuracy that can be expected for localization is already relatively poor due to the low numbers of photons that can be detected. This directly follows from the well-known fact that estimation accuracies worsen with decreasing photon count[9]. However, in many practical situations, low photon counts in the acquired data are unavoidable or necessary. In superresolution microscopy[3,5,6], relatively weak fluorophores often need to be chosen based on desirable attributes that are lacking in brighter fluorophores. Examples include the preferred use of weak genetically encoded fluorescent proteins for their labeling specificity[10], and the selection of weak dyes in multicolor superresolution imaging based on, for instance, the necessity of using spectrally well-separated fluorophores. Moreover, in live cell superresolution imaging and single molecule tracking, even the use of bright fluorophores typically results in low photon count images due to the very fast acquisition rate that is required to follow the dynamics of the cellular structures and molecules. More generally, in order to minimize phototoxicity and at the same time maximize the duration over which samples can be imaged[11] using the many conventional dyes and fluorescent proteins that have limited photostability, it might even be desirable to purposely acquire low photon count images using low excitation power levels. The extension of the imaging time is of particular importance for single molecule tracking, as substantially longer trajectories can potentially be observed.

Here we introduce UAIM, a method of low light imaging that enables localization with much higher accuracies than permitted by conventional imaging. In fact, the accuracy of UAIM can approach that of an ideal noiseless and unpixelated detector used under the same imaging conditions. UAIM uses an EMCCD detector at a high level of signal amplification, and in a highly unconventional setting in which the number of photons detected in each pixel of an acquired image generally (Supplementary Note 1) averages less than one (Supplementary Note 2). This may seem counterintuitive, as such an image (Fig. 1a) does not give a visually clear representation of the object of interest, as one would expect is required for obtaining high localization accuracies. We demonstrate with the localization of fluorescent beads that despite expectations, positional coordinates can be estimated with ultrahigh accuracies from such an image using the maximum likelihood estimator for EMCCD data (Online Methods). We further performed superresolution imaging and reconstruction of an Alexa647-labeled LAMP1+ cellular structure (Fig. 1b) where on average a relatively low 128.94 photons were detected per dye molecule. Finally, we tracked the plasma membrane dynamics (Fig. 1c and Supplementary Video 1) of an erbB2 receptor over an interval of just over 60 seconds, where on average 102.85 photons per image were detected from the Atto647N dye used to label the receptor.

Figure 1

UAIM and its applications

(a) Visual comparison of a UAIM image and a conventional EMCCD image. The UAIM (conventional) image is that of an Atto647N molecule from which ~72 (~84) photons per image were detected on average, and was acquired with an effective pixel size of 16 nm (253.97 nm) using a 1000× (63×) magnification. The brightest pixel of the UAIM (conventional) image has a mean photon count of 0.22 (21.85). The mesh representation, which displays intensity as height, contrasts more conspicuously the spiky appearance of the UAIM image with the relatively smooth appearance of the conventional image. Scale bars, 0.5 µm.

(b) Superresolution imaging of a LAMP1+ cellular structure using UAIM. Top, the widefield fluorescence image of the structure, formed by summing the 5063 UAIM (1000×) images from which single Alexa647 molecules were localized to produce a superresolution image. Bottom, the superresolution image constructed from the location estimates of individual Alexa647 molecules. The average number of photons detected per molecule is 128.94. Scale bars, 1 µm.

(c) Single molecule tracking using UAIM. Top, the trajectory of an erbB2 receptor (Supplementary Video 1) determined by localizing its Atto647N label from 594 UAIM (1000×) images. On average, 102.85 photons per image were detected from the Atto647N dye. Bottom left, one of the 594 UAIM images. The red box encloses the tracked Atto647N molecule. Bottom right, compacted (10 × 10-binned) version of the same UAIM image that facilitates visualization of the tracked Atto647N molecule (indicated by red arrow). Scale bars, 1 µm.

To demonstrate the substantial advantage that UAIM has over conventional EMCCD imaging in terms of localization accuracy, we took images of stationary 50 nm fluorescent beads using a microscope equipped with an EMCCD camera, and used maximum likelihood estimation to determine bead positions from both UAIM and conventional EMCCD images containing on average less than 200 bead photons. The standard deviations of the resulting estimates of the x positional coordinate of different beads are plotted in Figure 2a. Shown as a function of the mean photon counts detected per image from the beads, the standard deviations are clearly separated into a lower group which represents very high accuracies and corresponds to the beads imaged with UAIM, and a higher group which represents substantially poorer accuracies and corresponds to the conventionally imaged beads. The standard deviations for the beads imaged with UAIM range from 27.87 nm to 12.70 nm over a per image mean photon count range of 53.34 to 194.06, corresponding to a more than twofold improvement over the standard deviations for the conventionally imaged beads, which range from 63.11 nm to 38.07 nm over a per image mean photon count range of 79.64 to 145.13.

Figure 2

Experimental and theoretical demonstration of UAIM

(a) Comparing the standard deviations of the maximum likelihood estimates of the x coordinate of fluorescent beads imaged using UAIM (blue star) and conventional EMCCD imaging (blue circle). Each standard deviation corresponds to a different bead, identified by the mean number of photons detected from it per image. For each standard deviation, the corresponding limit of accuracy (UAIM: magenta star; conventional: magenta circle) is shown. Likewise, the corresponding ultimate limit of accuracy (UAIM: black star; conventional: black circle), which assumes an ideal detector that introduces neither noise nor pixelation, is shown. The UAIM (conventional) images were acquired with an effective pixel size of 16 nm (253.97 nm) using a 1000× (63×) magnification. (b) Theoretical analysis of point source localization. Decreasing the effective pixel size by increasing the magnification for EMCCD imaging at a high level of signal amplification (g = 1000) yields a limit of accuracy (star) that approaches the ultimate limit (blue line). The red markers at effective pixel sizes of 373.31 nm, 224.00 nm, and 160.00 nm (magnifications of M = 42.86, 71.43, and 100) correspond approximately to standard magnifications of 40× and 63×, and exactly to the standard magnification of 100×. For the same range of effective pixel sizes, the limit of accuracy (dot) corresponding to the common excess noise-based supposition, and the limit of accuracy (circle) corresponding to CCD imaging with a readout noise standard deviation of 2 electrons per pixel, are shown. (See Supplementary Note 9 for more details.)

For both UAIM and conventional EMCCD imaging we compared each standard deviation of x estimates to the corresponding theoretical best possible standard deviation, or limit of accuracy (Supplementary Note 3), and to the corresponding ultimate limit of accuracy (Supplementary Note 4) which assumes an ideal detector. In both cases, the standard deviations are reasonably close to their respective limits of accuracy, but only in the case of UAIM are these limits in turn very close to their corresponding ultimate limits (Fig. 2a). UAIM therefore allows estimation with standard deviations that approach what one can only achieve with an ideal detector. (For similar results obtained with the localization of stationary single molecules, see Supplementary Table 1 and Supplementary Note 5).

We confirmed our experimental results by carrying out maximum likelihood estimations on simulated images of a point source. The results obtained for an ideal, a conventional EMCCD, and a UAIM data set, each consisting of images of the same point source, are summarized in Table 1. Besides affirming that UAIM enables estimation with accuracies close to the ultimate limit, these simulation results suggest that the maximum likelihood estimator is capable of attaining the limit of accuracy in all cases.

Table 1

Results of maximum likelihood localizations performed on images of a point source simulated according to the ideal, conventional EMCCD, and UAIM imaging scenarios.

Imaging scenario	True valueof both x0and y0 (nm)	Mean ofx0, y0estimates (nm)	Limit ofaccuracy forboth x0 and y0(nm)	Standarddeviationof x0, y0estimates (nm)
Ideal	0	0.00, 0.00	6.19	6.29, 6.32
Conventional EMCCD	560	560.02, 559.80	11.72	11.71, 12.01
UAIM	560	559.90, 560.53	6.81	7.04, 6.83

Note that the single limit of accuracy shown per scenario applies to the estimation of both x and y, since the image of the point source was simulated to be exactly centered on a square region. See Supplementary Note 13 for the simulation parameters.

UAIM's stipulation of reducing the signal level per pixel is based on theoretical analyses that utilize our careful modeling[12] of the EMCCD signal amplification process. These analyses indicate that under the regime where each EMCCD pixel generally detects less than one photon on average, detector noise is minimized, and an image is produced that enables estimation of the quantity of interest with nearly as high an accuracy as an image that is free of detector noise (Supplementary Note 6). This regime was achieved here by decreasing the effective pixel size of the detector, via the use of a magnification about an order of magnitude higher (1000× for bead images, 900× for simulated images) than what is typical, thereby distributing the detected photons over many more pixels of the detector. Importantly, this approach not only minimizes the detector noise by virtue of the signal reduction per pixel, but generates a much more finely pixelated image of considerably higher resolution that more closely approximates an ideal image. By substantially reducing both major deteriorative effects of the detector, it yields estimation accuracies that nearly attain the ultimate limit of accuracy. To avoid the reduced field of view (Supplementary Note 7) that results from the use of high magnification, this approach might alternatively be implemented with a standard magnification, but using a non-standard EMCCD detector with unconventionally small pixels. (See Supplementary Note 8 for an altogether different approach to realizing UAIM.)

Summarizing the results of a theoretical analysis of our implementation of UAIM (Fig. 2b and Supplementary Note 9), we see that as the effective pixel size is decreased, the limit of the accuracy for estimating the positional coordinate of a point source improves and approaches the ultimate limit. At an effective pixel size of just 14.55 nm (1100× magnification), for example, UAIM yields a best possible accuracy of 6.74 nm, which is within a nanometer of the ultimate limit of 6.19 nm. In contrast, conventional EMCCD imaging at an effective pixel size of 160 nm (100× magnification), which is within the recommended size range[6] for fluorescence superresolution imaging, yields a best possible accuracy of 11.72 nm, nearly double the ultimate limit.

According to the common assertion[13] based on the signal amplification excess noise[14-16], the best estimation accuracy achievable with an EMCCD detector is worse, by a factor of √2, than that which is attainable with a hypothetical noiseless but pixelated detector. Our analysis (Fig. 2b) reveals a more complex picture, demonstrating that the √2 factor indeed approximates the EMCCD limit of accuracy well at large effective pixel sizes (standard magnifications) where the mean photon count per pixel is relatively high. However, at small effective pixel sizes (high magnifications) where the mean photon count per pixel is extremely low, the √2 factor underestimates considerably the attainable accuracy, which in fact approaches the ultimate limit.

A CCD detector, even one with a low readout noise level, is unsuitable for implementing UAIM (Fig. 2b). For such a detector, the limits of accuracy are no better than ~10 nm for effective pixel sizes between 400 nm and 160 nm (standard magnifications between 40× and 100×), and deteriorate quickly as the effective pixel size is decreased. This is due to the detected photons being overwhelmed by readout noise as their numbers in each pixel decrease with the pixel size reduction. As we explain in Supplementary Note 10, the same result (Supplementary Fig. 1) can be expected for the scientific complementary metal oxide semiconductor (sCMOS) detector.

We have demonstrated UAIM for image data with relatively low photon counts, but the principle of increasing the estimation accuracy by reducing the photon count in each pixel is applicable to a broad range of photon budgets (Supplementary Table 2 and Supplementary Note 11). In addition, though we have applied UAIM to super-localization microscopy, it is a general imaging method that can also be of benefit in other applications (e.g., astronomy[17], computer vision[18]) in which quantities (not limited to the positional coordinates of objects) need to be extracted from the acquired images. Importantly, the fundamental nature of UAIM is such that it can in principle be incorporated into existing imaging techniques to potentially substantially improve the obtainable accuracies.

Online Methods

Microscope setup and image acquisition

All imaging experiments were carried out on a Zeiss Axiovert 200 microscope (Carl Zeiss). The sample was imaged with a Zeiss Plan-Apochromat oil immersion objective lens (see experiment-specific details below). For UAIM, three Zeiss external optovars (two 2.5× and one 1.6×) concatenated to achieve a tenfold further magnification were positioned in front of an Andor iXon DU-897 EMCCD camera (Andor Technologies). For conventional EMCCD imaging, the external optovars were removed. The electron multiplication gain of the camera was set to 950.

For imaging cell samples, a Zeiss dual video adaptor was installed at the side port of the microscope, which split the emission into two light paths. The external optovars and the Andor camera were attached to one of the light paths, and a second (i.e., reference) Andor iXon camera was attached to the other light path. The latter camera was used to image fiduciary markers for drift correction, and also aided in the focusing/scanning of the sample.

For imaging beads, a 405 nm laser (Opto Engine) was passed through an HQ405/30× cleanup filter (Chroma Technology) and reflected by a 455DCLP dichroic mirror (Chroma Technology) to illuminate the sample, and the resulting fluorescence signal collected by the objective lens (100×, 1.4 NA for UAIM (for a total magnification of 1000×); 63×, 1.4 NA for conventional EMCCD imaging) was passed through an HQ480/40m CFP emission filter (Chroma Technology).

For imaging cells, light from either a 635 nm laser (Opto Engine) or a 488 nm laser (Blue Sky Research) was reflected by a quad-edge dichroic (Di01-R405/488/543/635-25×36; Semrock) to illuminate the sample, and the resulting fluorescence signal was passed through a quad-band emission filter (FF01-446/515/588/700-25; Semrock). Prior to entering the microscope, the 635 nm laser line was passed through a cleanup filter (LD01-640/8-12.5; Semrock) and a quarter wave plate, and the 488 nm laser line was passed through two cleanup filters (10LF10-488; Newport).

For superresolution imaging, the cell sample was illuminated with the 635 nm laser. During imaging, the 488 nm laser was turned on either every 76th frame (for LAMP1 imaging) or every 101st frame (for cell membrane imaging) to photoactivate Alexa647. A 100×, 1.4 NA objective lens was used for LAMP1 imaging at a total magnification of 1000×, and a 63×, 1.46 NA objective lens was used for cell membrane imaging at a total magnification of 630×.

For single molecule tracking and stationary single molecule imaging, the sample was illuminated with the 635 nm laser. For single molecule tracking and the UAIM imaging of stationary single molecules, a 100×, 1.4 NA objective lens was used to achieve a total magnification of 1000×. For the conventional EMCCD imaging of stationary single molecules, a 63×, 1.4 NA objective lens was used.

The camera acquisition parameters for all imaging experiments are given in Supplementary Note 12.

Sample preparation

A bead sample was prepared by adsorbing a dilute solution of 50 nm yellow-green beads (Polysciences) on poly-lysine coated glass cover slips. The beads were imaged in ultrapure, filtered water.

The superresolution imaging of LAMP1+ structures and of the cell membrane was carried out with HMEC human endothelial cells and Z310 rat epithelial cells[19], respectively. The cells were plated on Zeiss high performance cover slips and a day before fixation, the cells were pulsed with 100 nm yellow-green beads (Polysciences) to allow internalization of the beads. The beads were used as fiduciary markers for drift correction.

For LAMP1 staining, the sample was fixed with 1.7% (w/v) paraformaldehyde (PFA) and permeabilized with 0.02% (w/v) saponin, each for 10 min at room temperature; then blocked with 4% BSA (w/v) in PBS, stained with mouse anti-LAMP1 antibody (1:100 dilution; Developmental Studies Hybridoma Bank; clone # H4A3) and then counter-stained with goat anti-mouse IgG Alexa647 (1:1000 dilution; Life Technologies), each for 25 min at room temperature.

For cell membrane staining, Z310 cells were grown to confluence, then stained with 1 µg/ml cholera toxin B subunit labeled with Alexa647 (Life Technologies) for 15 min on ice and fixed with 1.7% (w/v) PFA for 15 min on ice.

For both types of samples, the medium in the cells was changed to an imaging buffer (PBS pH 7.4 containing 10% (w/v) glucose; 50 mM 2-mercaptoethylamine; 0.5 mg/ml glucose oxidase; 40 µg/ml catalase) just before the start of imaging.

Live cell single molecule tracking experiments and the imaging of stationary single molecules were carried out with BT474 human epithelial cells. An antibody fragment (Fab) against the human erbB2 receptor labeled with Atto647N dye was pulsed at a concentration of 50-270 pM. For tracking experiments, the cell sample was imaged at 37°C. For the imaging of stationary single molecules, the cell sample was fixed with 1.7% (w/v) PFA for 10 min at room temperature and then imaged at room temperature in PBS containing 1% (w/v) BSA.

Modeling the image of a point source

The image of a point source is assumed to be described by the Airy point spread function[20]. Expressed in the form of an image function[21], which assumes unit magnification and the point source to be located at the origin of the object space in which it resides, it is given by where n is the numerical aperture of the microscope objective, λ is the wavelength of the detected photons, and J is the first order Bessel function of the first kind.

Image simulation

As is typical, photon detection is modeled as a Poisson process. Therefore, given that on average N photons per image are detected from a point source over the entire detector plane (i.e., R), an ideal image of arbitrarily high resolution with no detector noise is simulated by first generating N pairs of x and y coordinates, where N is the number of detected photons drawn from the Poisson distribution with mean N. Each (x, y) pair represents the location at which a photon is detected, and is generated as random numbers drawn from the probability distribution characterized by the probability density function[21] which is just a scaled and shifted version of equation (1) that accounts for the magnification M of the microscope setup and the x and y positional coordinates x and y of the point source in the object space where it resides. Any (x, y) pair that places a photon outside the chosen finite region C ⊂ R2 occupied by the detector is discarded, leaving N pairs of x and y coordinates within C that correspond to photons detected from the point source. The number of background photons (i.e., photons detected from anything other than the point source) in an image is likewise Poisson-distributed, and is assumed to be independent of the number of photons detected from the point source. Therefore, given that on average β background photons are detected per image, N additional (x, y) pairs, where N is a number drawn from the Poisson distribution with mean β, are generated. Assuming a uniform background for the image, each background (x, y) pair is generated as random numbers drawn from the spatial distribution characterized by the probability density function b(x, y) = 1/A, (x, y) ∈ C, where A is the area of the region C. The simulated ideal image is thus composed of a total of N = N + N pairs of x and y coordinates.

To simulate a conventional EMCCD or a UAIM image of K pixels, the mean ν, k = 1, 2, …, K, of the Poisson-distributed number of photons detected at the kth pixel is first calculated as where N and f(x, y) are as defined above, C is the region in the detector plane occupied by the pixel, and β is the mean number of background photons at the pixel. Given ν, the eventual data at the kth pixel is then simulated as a random number drawn from the probability distribution characterized by the probability density function[12] where η and σ are the mean and standard deviation of the Gaussian random variable used to model the EMCCD camera's readout noise at the kth pixel, and g is the signal amplification level (i.e., electron multiplication gain) of the EMCCD camera.

For the parameter values used to simulate the ideal, conventional EMCCD, and UAIM data sets of Table 1, see Supplementary Note 13.

Maximum likelihood estimation

Given an ideal image consisting of N photons detected at positions {r = (x, y) ∈ C : j = 1, …, N}, where C is the finite region in the detector plane occupied by the detector, the maximum likelihood estimation of the x and y coordinates of the point source was carried out by maximizing the log-likelihood function where the probability density function f is given by equation (2), A is the area of the region C, and N and β are as defined above.

As indicated by the method of simulation, the data collected in each pixel of an EMCCD detector is modeled as the sum of an amplified Poisson signal and a Gaussian random variable representing the readout noise. As such, given a conventional EMCCD or a UAIM image of K pixels, be it simulated or experimental, the maximum likelihood estimation of the x and y coordinates of the point source was carried out by maximizing the log-likelihood function where for k = 1, …, K, z is the data at the kth pixel, and p is the probability density function given by[12,13,17] where I is the first order modified Bessel function of the first kind, and all parameters are as defined above. Instead of equation (4), equation (7) was used for estimation because it can be computed more efficiently, and has been demonstrated to be an excellent approximation of equation (4) at high signal amplification levels[12].

Maximization of the log-likelihood functions of equations (5) and (6) was in each case implemented as the minimization of the negative of the log-likelihood function using the fminunc function of the MATLAB optimization toolbox (MathWorks). In the analysis of all data sets (simulated and experimental), only the parameters x and y were estimated. All other parameters had fixed values, some of which were first estimated by other means in the case of experimental data (Supplementary Note 14). For all data sets, limits of the localization accuracy were computed (Supplementary Note 15) either for comparison with the standard deviations of the maximum likelihood estimates of x and y, or for characterizing the accuracy with which a representative maximum likelihood localization was carried out.

Comparison of UAIM and conventional EMCCD imaging

Fluorescent beads were chosen for our experimental comparison of conventional EMCCD imaging and UAIM for their photostability, which importantly allowed us to obtain the necessary number of repeats of the image of a given bead for determining the localization accuracy. A total of 10 conventional EMCCD (63×) and 11 UAIM (1000×) data sets were analyzed, each consisting of 951 to 1081 images of a single bead. For a given bead, the center of its image was approximately identified by eye and a square region of interest (ROI) (13 × 13 (61 × 61) pixel array for conventional EMCCD (UAIM)) was chosen around it. The selected ROI was extracted from each image of the bead, and individually subjected to a maximum likelihood estimation of the bead's positional coordinates x and y. To correct for drift that was observed during the imaging of the bead, best-fit lines to segments of the x estimates plotted as a function of time were subtracted from the x estimates. The segments were identified based on drift patterns visually observed in the x estimates versus time plot. Independently, the y estimates were piecewise linearly drift-corrected in like manner. The standard deviations of the drift-corrected x and y estimates were then computed and respectively used as a point in the plots of Figure 2a and Supplementary Figure 2.

The same procedure was followed to compare the two imaging methods using the localization of single molecules. Stationary erbB2 receptors were imaged with Atto647N-labeled anti-erbB2 Fab. From the resulting images, 2 conventional EMCCD (63×) and 2 UAIM (1000×) data sets were selected, each consisting of between 301 and 403 images of a visually identified, well-isolated Atto647N molecule. To ensure that a single dye molecule was being observed in each data set, single-step photobleaching was visually verified for the molecule. The images in a data set consisted of a consecutive sequence of frames over which the Atto647N molecule maintained a relatively stable photon count level, as determined by a plot of the sum of the molecule’s ROI pixel values over the frames. The results are presented in Supplementary Table 1 and discussed in Supplementary Note 5.

Superresolution data analysis

To demonstrate superresolution imaging with UAIM, two different superresolution data sets, one of a LAMP1+ cellular structure immunofluorescently stained with Alexa647, and one of a cell membrane stained with Alexa647-labeled cholera toxin B subunit, were respectively acquired with a 1000× and a 630× magnification, and were analyzed as follows. Image frames at which the 488 nm activation laser was turned on were excluded from analysis, since they contained a dense distribution of activated Alexa647 molecules. To further ensure that only images with sparsely distributed activated Alexa647 molecules were analyzed, an additional 14 (15) frames immediately following each activation frame were discarded from the LAMP1+ cellular structure (cell membrane) data set. To facilitate the identification of Alexa647 spots, the remaining frames in the data set were compacted by a 10 × 10 binning of the pixels to produce images that appear as if they were acquired using a standard magnification that was a factor of 10 lower. Alexa647 spots in the compacted frames were then identified using a wavelet-based spot detection algorithm[22].

Each identified spot in the compacted frames was represented by a 5 × 5-pixel ROI that was centered on the spot's brightest pixel. Some of these ROIs were eliminated from further analysis as follows. To select only the relatively well-isolated Alexa647 spots in each image frame for maximum likelihood localization analysis, ROIs that overlapped with more than one other ROI were discarded. Furthermore, of the remaining ROIs that overlapped with only one other ROI, both ROIs of an overlapping pair were retained for further analysis only if they overlapped by no more than 5 pixels. Otherwise, only the ROI with the brighter center pixel was retained.

Each selected ROI was then subjected to a nonlinear least squares estimation to obtain estimates for the expected photon count N detected from the Alexa647 molecule, and the expected per-pixel background photon count β (Supplementary Note 14). The histogram of the majority of the photon count estimates exhibited an approximately Gaussian distribution. ROIs whose photon count estimate fell outside of this Gaussian distribution (outside the range of 25 to 350 photons for the LAMP1+ cellular structure data set, and 50 to 450 photons for the cell membrane data set) were eliminated from further analysis. Similar elimination of ROIs was done based on the background estimates, the majority of which also formed an approximately Gaussian distribution. (ROIs with a background estimate outside the range of 0.05 to 1 photon per pixel in the LAMP1+ cellular structure data set, and outside the range of 0.6 to 1.3 photons per pixel in the cell membrane data set, were discarded.) In total, the eliminations based on the photon count and background estimates resulted in 4.4% (19.4%) of the Alexa647 spots being discarded for the LAMP1+ cellular structure (cell membrane) data set.

The Alexa647 molecules that survived elimination were subjected to maximum likelihood estimation to obtain the x and y positional coordinates for the construction of the superresolution image. The localization of each Alexa647 molecule was performed on a 50 × 50-pixel ROI extracted from the original, uncompacted image data. The 50 × 50-pixel ROI was obtained by mapping the pixel coordinates of the molecule’s 5 × 5-pixel ROI back to the pixel coordinates of the original data.

The x and y maximum likelihood estimates of an Alexa647 molecule were discarded if they placed the molecule outside of its ROI. Furthermore, estimates were discarded if the function fminunc returned an exit flag of less than 1. The x and y estimates that remained were then corrected for drift as follows. From the data set that was acquired by the reference camera during the acquisition of the superresolution data, 3 beads were visually identified from the frames at which the 488 nm activation laser was turned on. For each bead, an ROI centered on its image was extracted from each of the activation frames. (For each bead, the same ROI was in fact extracted from each activation frame, as the drift was not visually detectable.) Each ROI was then subjected to a nonlinear least squares localization where the point spread function model was assumed to be a 2-dimensional Gaussian function.

Since images acquired by the reference camera were not perfectly spatially aligned with the images produced by the camera that captured the superresolution data, the resulting x and y estimates for the 3 beads were multiplied by a transformation matrix to convert their values to the coordinate system of the superresolution data set. The transformation matrix was determined using a separate data set where a bead sample was simultaneously imaged by the two cameras. Multiple beads that spanned the field of view of the camera that captured the superresolution data were chosen, and their ROIs were extracted from the images acquired by each camera, and subjected to independent localizations using a nonlinear least squares estimator where a Gaussian point spread function was assumed. The resulting pairs of x and y estimates for each bead were then used to infer the spatial transformation. Specifically, the function cp2tform of the MATLAB image processing toolbox was used to obtain a nonreflective similarity transformation matrix.

To determine the drift along the x dimension in the superresolution data, the transformed x estimates for each of the 3 reference camera beads were stored in a vector in order of ascending time, and a nonlinear least squares optimization was used to globally fit a cubic polynomial to the 3 vectors corresponding to the 3 beads. The resulting drift curve was then subtracted from the x maximum likelihood estimates of the Alexa647 molecules to correct them for drift. Drift in the y dimension was corrected for in completely analogous fashion.

For the construction of the superresolution image, the pixe size was chosen to be the effective pixel size of the original data set (16 nm for the LAMP1+ cellular structure data set, and ~25.4 nm for the cell membrane data set). The dimensions of the image were also chosen to be the same as an original data image (200 × 512 (512 × 512) pixels for the LAMP1+ cellular structure (cell membrane) data set).

The superresolution image was generated by a simple binning of pixel counts. Specifically, starting with a count of 0 in each pixel of the image, the value of a pixel was incremented by 1 for every (drift-corrected) x and y pair that positions an Alexa647 molecule within the pixel. For presentation (Fig. 1b for the LAMP1+ cellular structure data set, Supplementary Fig. 3 for the cell membrane data set), the superresolution image was adjusted linearly using the imadjust function of the MATLAB image processing toolbox with no user-supplied adjustment parameters. A total of 11,593 (23,721) x and y pairs were used to construct the LAMP1+ cellular structure (cell membrane) superresolution image, and these location estimates were the result of the analysis of 5,063 (4,909) data images. The same data images were also summed to produce a widefield fluorescence image for visual comparison with the superresolution image. The limits of accuracy computed for each superresolution data set are given in Supplementary Table 3.

Live cell single molecule tracking data analysis

To demonstrate tracking with UAIM, erbB2 receptors moving in the plasma membrane of live BT474 human epithelial cells were imaged using Atto647N-labeled anti-erbB2 Fab at a 1000× magnification, and a data set was analyzed as follows. To facilitate the visualization of the data, the acquired images were compacted by a 10 × 10 binning of the pixels to produce images that appear as if they were acquired using a standard magnification of 100×. By manually scrolling through this compacted version of the data in the image viewer of our custom microscopy image analysis software[23], the track of an Atto647N molecule was identified by eye. In each of the compacted track frames, a 5 × 5-pixel ROI was manually centered on the tracked Atto647N spot using a tool that is part of the same custom software environment as the image viewer. Maximum likelihood localization of the tracked molecule in each frame was then carried out on a 50 × 50-pixel ROI from the original, uncompacted image data, obtained by mapping the pixel coordinates of the 5 × 5-pixel ROI back to the pixel coordinates of the original data. The resulting x and y coordinates were used to generate the track shown in Figure 1c and Supplementary Video 1, and the associated limits of accuracy are given in Supplementary Table 3.