Dynamic quantitative phase imaging for biological objects using a pixelated phase mask.

This paper describes research in developing a dynamic quantitative phase imaging microscope providing instantaneous measurements of dynamic motions within and among live cells without labels or contrast agents. It utilizes a pixelated phase mask enabling simultaneous measurement of multiple interference patterns derived using the polarization properties of light to track dynamic motions and morphological changes. Optical path difference (OPD) and optical thickness (OT) data are obtained from phase images. Two different processing routines are presented to remove background surface shape to enable quantification of changes in cell position and volume over time. Data from a number of different moving biological organisms and cell cultures are presented.

1. Introduction

The ability to instantaneously measure live cells and follow motions and processes over time provides valuable information to researchers studying cellular dynamics, motility, and cell and tissue morphology (see, for example, [1-3]). Quantitative phase imaging can measure structures using interference images analogous to those viewed with phase-contrast, differential interference contrast and digital holography [2]. Phase images reveal features and quantitative data that cannot be easily obtained using conventional brightfield optical imaging.

Phase imaging quantifies optical path difference (OPD) variations due to small variations in refractive index and thicknesses of different structures and materials within cells and tissues [2,4]. Quantitative phase measurements can be directly combined and correlated with any other type of microscopic imaging such as fluorescence imaging. To obtain phase image data, harmless light levels are used, and samples do not need to be stained, labeled or marked. By taking short snapshots in rapid succession, the dimension of time opens up the ability to track motions of cells, see how cells interact with one another, and follow small motions within cells, tissues and structures.

Techniques developed for full-field phase-imaging interference microscopes have historically relied upon temporal phase-measurement methods that obtain interferograms sequentially, and therefore require good vibrational damping with static specimens in order to obtain high-quality data [5]. These techniques have been underutilized for biological measurements because they are sensitive to motion and vibration [1-3,6-9]. The technique presented here incorporates a pixelated phase mask enabling all data to determine phase to be captured in a single exposure that is a fraction of a millisecond [10-13].

Commercial interferometric microscopes generally employ “narrowband” illumination with temporal coherence lengths of tens of microns (wavelength bandwidths of 10–50 nm) or white light “broadband” sources with temporal coherence lengths as short as 1 µm [5]. These types of sources help to reduce effects of reflections off of nearby surfaces and speckle in the imaging system. However, the shorter the temporal coherence length, the more important it is to match the optical path lengths of the reference and test beams [14]. Further improvement in microscope imaging systems has been obtained by incorporating extended low spatial coherence sources (sources with extended areas) [5]. When measuring biological samples, this enables isolating a plane of interest in space without getting spurious interference patterns from the coverslip and other nearby surfaces [15].

This paper presents current research in developing an interference microscope specially designed for measurement of living and moving biological samples in liquids under coverslips. For these examples, objects are viewed in double pass on a reflective surface. The method is extensible to measurement in transmission, or for use with immersion objectives. Here we discuss the design of this instrument and techniques that have improved image quality as well as processing methods to remove background surface shape enabling isolation of objects from the background for extraction of position and morphological information.

2. Dynamic interference microscope

The interference microscope used for this work is based upon a Linnik configuration (Fig. 1 ) [5,15]. It is comprised of a Köhler-type illumination system, and a simple imaging system as shown in Fig. 1. An aperture stop enables controlling the size of the source in the entrance pupil of the microscope objectives, while a field stop enables easier alignment of the system.

Fig. 1

Optical schematic and photograph of dynamic interference microscope.

The incoming illumination passes through a polarizing beam splitter before the microscope objectives creating orthogonal linearly polarized test and reference beams. The relative irradiances of the object and reference beam can be balanced for maximum contrast by adjusting the angle of the polarizer. A quarter-wave plate (QWP) before the camera converts the two polarized beams to right- and left-handed circular polarization to produce interference fringes at the pixelated phase mask. For the measurements in this paper, samples in water or other liquids are placed on a mirror under a cover slip providing a double pass through the object. Sources with wavelengths of 660 nm and 785 nm were used with 20× NA 0.45, or 50× NA 0.8 objectives. The imaging “tube” lens magnification (combination of tube and FOV lenses) varies from 1× to 2.25×.

Figure 1 (right) shows a photograph of an engineering brassboard system. The Linnik objective is seen below the microscope (blue and white box) with a 5-axis translation stage beneath it for adjusting position and tip/tilt of the sample. This compact design enables it to be used on a variety of stands so that it can be interfaced with different types of staging and cell handling systems.

2.1. Extended low coherence source

Current practice in interference microscopes is to use a source with a short temporal coherence length like an LED (light emitting diode). Obtaining high quality interference fringes with these types of sources requires tight tolerances on the optical system to equalize path lengths [14,16]. A major requirement is that the spatial and temporal coherence functions need to overlap one another for high quality interference fringes [17,18]. Dispersion induced by the coverslip, transparent object, and liquid, will shift the spatial coherence function relative to the temporal coherence function and can reduce fringe modulation. One way of dealing with this is to put an equivalent amount of glass and liquid in the reference arm as in the test arm to equalize optical path lengths [19], or change the illumination system to account for it [17-19].

Another way to get around this limitation is to tailor the temporal and spatial coherence functions to accommodate unmatched dispersive media in both arms and still have the positive effects of low coherence [17,18]. This is done by designing the source so that the temporal coherence length is long enough to account for the path length differences generated by the dispersive media (the cover slip and liquid), while reducing the spatial coherence to localize the interference fringes in space, so that spurious reflections and coherent noise are not present.

To reduce the spatial coherence to 25–35 µm, diode laser sources having temporal coherence lengths of 250–300 µm (spectral bandwidths of ~1.5 nm) are focused onto a rotating diffuser [15] and then coupled into an NA 0.2 multi-mode optical fiber with a 1000 µm core. The laser has sufficient temporal coherence over path length differences that take into account the cover glass, while the spatial size of the source given by the multi-mode optical fiber will limit the spatial coherence.

2.2. Optimizing fringe visibility

For a Linnik interference microscope objective, the reference objective and the reference mirror are moved together to match the path lengths in the reference and test arms to maximize fringe contrast. The reference mirror remains in a fixed position relative to the reference objective. When imaging an object, focus is first adjusted on the sample. The entire reference arm (reference microscope objective and reference mirror) is moved to maximize fringe contrast. As long as there is the same coverslip and liquid thickness, the reference path length should not need to be readjusted. Further optimization of interference fringe visibility is accomplished by balancing the amount of light in the reference and test beams by rotating the polarizer in the illumination system.

Because the microscope objectives used are not designed for coverslips, the coverslip and liquid thickness will generate some spherical aberration in the measurements. However, because it is possible to subtract out small amounts of spherical aberration from measurements, there is no noticeable image degradation with numerical apertures ~0.5. At 20× and ~200–300 µm total thickness, very good fringe contrast is easily obtainable. At 50×, it is very important to match aberrations of the two microscope objectives [20], have a high quality beam splitter with very little OPD error [14], and align these components to maximize image quality and fringe contrast [21].

3. Obtaining phase images and determining optical thickness

Pixelated micropolarizer arrays were first used for imaging polarimetry [22]. They were adapted for use in phase measurement interferometry in the last decade [23-25]. For phase imaging applications, pixelated phase mask technology enables single frame phase measurement, and permits the use of a wide variety of wavelengths and source bandwidths [26]. All necessary information to determine phase is recorded in a single snapshot so that vibration isolation, phase shifting, or scanning through focus are not needed.

3.1. Phase imaging with a pixelated phase mask

Wire-grid polarizers are made of tiny metal wires that are deposited on a transparent substrate (typically aluminum wires on a glass substrate) as shown in Fig. 2 (left). The linewidth, thickness and period of the wires are approximately 100nm, 120nm and 240nm respectively. These sub-wavelength structures have the property of reflecting light polarized parallel to the wires and transmitting light polarized perpendicular to them. They function as efficient polarizers over a wide range of wavelengths and angles with a specified contrast ratio of 50:1 [26]. The polarization mask used for this instrument is constructed from an array of wire grid micropolarizers as shown in Fig. 2. The micropolarizers are oriented in the plane of the mask at angles yielding relative phase shifts between right- and left-handed circularly polarized object and reference beams of .

Fig. 2

(left) Scanning electron micrograph (SEM) image of a wire grid polarizer (Courtesy of Moxtek). (right) Illustration of the pixelated phase mask that is bonded to the detector array having 4 different polarizations comprising a unit cell of 2 × 2 different relative phase values.

At the phase mask, the reference and test beams have orthogonal circular polarizations (i.e., right-hand circular and left-hand circular). When the two beams are combined, the measured irradiance at each pixel of the camera is given by Eq. (1) [27],

where and are the irradiance of the object and reference beams respectively, is the phase shift between the object and reference beams induced by the micropolarizer at angles with respect to the x, y plane, and is the total phase difference between the object and reference beams for the double pass through the object reflecting off the mirror. When this equation is applied to each of the 4 pixel types in the unit cell, shown in Fig. 2, phase differences of 0°, 90°, 180°, and 270° are encoded into interferograms that can be written as Eqs. (2)–(5):

From a single image, four simultaneous full-field interferograms are synthesized by combining pixels of each phase type (A, B, C and D). These four interferograms can be processed by a variety of algorithms that are well-known for calculating image phase [4,28]. Using the well-known four-frame phase algorithm, the double pass phase variation is written in Eq. (6) as

where ATAN2 is the 2π arctangent function. This produces a modulo 2π (wrapped) phase map which then needs to be unwrapped using standard techniques [29,30]. These phase image calculations with unwrapping can be done in real-time [31].

3.2. Determining OPD and optical thickness

Each interference fringe in reflection corresponds to one-half wave of optical path difference (OPD) given by

Typically, the raw units of OPD are in terms of wavelengths of the source light (waves). When measuring transparent objects such as glass, optical coatings and liquid media, optical metrologists often refer to these OPD variations as optical thickness (OT). Whereas OPD is typically presented in units of waves (or wavelengths), but can also have the units of length, OT has units of length. We are defining optical thickness as

where and are the integrated optical path lengths, , for the object and reference beams as defined in Ref. [32]. For quantitative phase imaging OPD and OT can be used interchangeably. OT and OPD values are the actual physical thickness of a transparent object. They both incorporate the integrated OPL where the refractive index and physical thicknesses are coupled. These two quantities cannot be separated without a priori information about one or the other.

For the methods presented in this paper, transparent biological objects are surrounded by liquids such as water and placed between a mirror and a cover slip (see Fig. 3 ). Thicker areas of the object or areas with higher refractive indices will yield larger OT values as shown in the profile on the bottom of Fig. 3.

Fig. 3

Optical path length through test sample includes the coverslip, liquid and objects. Graph (bottom) shows an optical thickness profile for the section within the highlighted area.

For the examples shown in this paper, we are presenting the measured OT or OPD. We have not assumed any refractive index or cell thickness values. Because of this, the vertical scales appear to be compressed. If an object has a physical thickness of 10µm and has a refractive index variation of 0.05 greater than the liquid surrounding it (e.g. an object index of 1.38 in water of index 1.33), then the OT will have a value of 500 nm peak to valley relative to the background, while the OPD would be 0.75 waves peak to valley for a wavelength of 660 nm.

3.3. Following motion with dynamic measurements

To illustrate dynamic quantitative phase-image movies, subtle changes in water temperature were measured across a 50 mm field of view using a low coherence Twyman-Green interferometer with a pixelated phase camera in a horizontal configuration. The object was a plexiglass cuvette with 5 mm thick walls and a chamber 20 × 35 × 5 mm. The cuvette was filled with warm water and a drop of cold water was added using a dropper. Figure 4 presents a series of 4 images taken 0.1 sec apart as the drop enters the cuvette. Figure 4(a) shows the interference fringes. The drop can be seen at the top of the first image and diffuses throughout the other frames. The fringes represent the phase deviation owing to a slightly smaller refractive index for the cold water than that of the warm water. Figure 4(b) shows the wrapped (modulo 2π) phase obtained from Eq. (6). Discontinuities in these images are due to the modulo 2π nature of the 2π arctangent function (ATAN2).

Fig. 4

Frames from a time series of a drop of cold water dropped into cuvette of warm water. (a) Interference fringes. (b) Wrapped phase modulo 2π. (c) Phase images with a range (high-low) of 1.75 waves. (d) Cross-section profile of horizontal line through center of 2nd phase image in (c).

The final step of the phase measurement process is to unwrap the phase and remove these 2π ambiguities. This is done using an algorithm that traces a path and adds or subtracts multiples of 2π until differences between adjacent pixels are less than π (see for example [29,30]). Figure 4(c) shows the phase images where the color scale from high (warm) to low (cold) goes from red to orange to yellow to green to blue. Blue is a shorter optical path difference (OPD) relative to red. With the quantitative phase map it is easy to discern the water drop in each image. The pseudo-color range of this phase measurement from maximum to minimum is 1.75 waves OPD at 660 nm or about 1.15 µm of OT. Figure 4(d) shows a profile through the center of the second phase image in Fig. 4(c) with scales for both optical thickness (OT) and OPD (phase).

3.4. Optical performance and measurement sensitivity

For our initial measurements, tube lenses with different magnifications were inserted to optimize field of view (FOV) and optical resolution on a camera that had 9µm square pixels. Because there is always a tradeoff between getting the maximum FOV and the best optical resolution, the system design was modified to have two different fields of view (FOV) using a flip-in FOV lens in the imaging arm. This does not change the optical resolution of the microscope, but enables choosing between using a large FOV, or maximizing resolution by changing the total system magnification by a factor of 2.25×. With 20× objectives, this yields 20× and 45×. The system was further improved by utilizing a camera with a smaller pixel pitch of 7.4 µm and larger sensing area of 9.2 × 8.88 mm yielding a 20× FOV of 460 × 440 µm and 204 × 197 µm at 45×.

With a 20× NA 0.45 Linnik, the optical resolution is 0.9 µm (0.61 λ/NA). The sampling at the detector with the 1× tube lens is 0.37 µm and with the 2.25× FOV lens, it is 0.16 µm, indicating that with 20× total system magnification the image is sampled close to Nyquist, and with 45× it is slightly oversampled. With a 50× NA 0.8 Linnik the optical resolution is 0.5 µm with FOV’s of 230 × 170 µm at 50× and 100 × 75 µm at 112.5×. The sampling intervals respectively are 0.15 and 0.066 µm providing oversampled images.

Lateral and vertical distances were calibrated using a NIST-traceable VLSI optical profiler step height standard certified to be 44.1 nm ± 0.9 nm [33]. Since accuracy is determined relative to a standard, this calibration is what limits the measurement accuracy. The phase sensitivity and instrument noise were determined by taking two consecutive phase measurements without an object present. The rms of the point-by-point difference between the two measurements indicates the measurement repeatability and provides a standard measure of the instrument noise level [34]. For this instrument, the rms repeatability for single measurements is ~1.0 nm. This means that the spatial phase sensitivity between pixels in the same measurement is ~1.0 nm, and the temporal phase sensitivity over short periods of time will also be ~1.0 nm. The performance over longer periods of time depends upon the thermal and mechanical stability of the environment.

3.5. Effects of coherent and incoherent imaging

Earlier in this paper we discussed the importance of tailoring the spatial and temporal coherence length for this system. Figure 5 illustrates the effect of using a rotating diffuser to improve illumination. Speckle from a 660nm diode laser source with a 300µm temporal coherence length can be seen in the irradiance (Fig. 5(a)) and the phase (Fig. 5(c)). This temporal coherence length ensures we are not seeing interference from nearby surfaces, but there is still obviously coherent noise present. Figure 5(b) and Fig. 5(d) show improvement to the irradiance and phase by introducing a rotating diffuser. When the diffuser is rotated at a high speed, speckle will be averaged, thereby eliminating coherent noise and generating a uniform extended source. The illumination falls off in Fig. 5(a) because the illumination system is designed assuming a NA 0.2 input that is not completely filled without using the diffuser. The engineered diffuser’s scattering angle fills the entire NA 0.2 and therefore more modes of the multi-mode fiber. The effective size of the source at the output of the optical fiber enables tailoring of the spatial coherence.

Fig. 5

Improved imaging by reducing spatial coherence using a rotating diffuser. Irradiance of a VLSI step height standard phase object (a) without diffuser, and (b) with diffuser. Measured phase of object (c) without diffuser and (d) with diffuser. (e) Comparison of a phase line profile through dashed lines in (c) without diffuser and (d) with diffuser showing improvement by reducing spatial coherence.

The use of a rotating diffuser is typical in commercial interferometers for optical testing such as Fizeau and Twyman-Green interferometers [35], but has been underutilized in interference microscopes. This is because interference microscopes typically use temporally less coherent sources than those required to deal with biological object dispersion. The use of the rotating diffuser in this instrument improves image quality and provides a spatially incoherent image as seen in Fig. 5. The improvement is even more noticeable in phase measurements. Figure 5(e) shows a single cross section profile through the phase measurements shown in Figs. 5(c) and 5(d). The noise in the phase is significantly reduced with the rotating diffuser.

4. Sample measurements

4.1. Types of images obtainable from polarization phase data

The pixelated phase mask sensor enables a number of different types of images to be obtained simultaneously as illustrated in images of a protozoa in Fig. 6 . When the values of all 4 types of pixels from Fig. 2 are averaged a brightfield image is obtained (Fig. 6(a)). When values from one type of pixel are displayed, an interferogram or phase-contrast image is obtained (Fig. 6(b)). To simulate images that look similar to dark field images, the phase gradient magnitudes (PGM) are obtained by calculating (Fig. 6(c)) [36]

Fig. 6

Images of a protozoa determined from pixelated phase data. (a) Brightfield (irradiance/intensity). (b) Phase contrast (interference – a single interferogram). (c) Phase gradient magnitudes (simulated dark field). (d) Simulated DIC (x gradient). (e) Pseudo-colored optical thickness (from phase). (f) 3D optical thickness (from phase).

Simulated DIC (differential interference contrast) images are obtained by calculating the gradient of the phase in the x or y direction (Fig. 6(d)). Combining all 4 pixels using Eqs. (2)–(9) produces either a phase image or optical thickness map. Pseudo-colored contour and 3D representations of the calculated optical thickness are shown in Figs. 6(e) and 6(f). These were taken at 50× with a 660 nm source. The optical thickness (OT) of this sample is about 375 nm maximum (red) to minimum (blue)(peak-to-valley or P-V). This is not the physical thickness of the protozoa. Because these are relative and not absolute measurements, we have arbitrarily set the minimum value to zero. Note that internal structures are readily visible.

4.2. Dynamic measurement of moving biological objects

Dynamic measurements are made using short exposures on the order of 0.5–1 ms with user specified time delays between frames of data ranging from no delay to several hours. The current maximum achievable acquisition rate is 25 frames per second (fps) in full frame mode. The data are processed to obtain unwrapped phase, and then movies can be compiled from the time series of images. To produce the objects for many of these examples, water from a nearby culvert was cultured to produce a number of different species of simple pond organisms [37]. Examples of the motion of cilia in a large paramecium are shown below in Fig. 7 and the Media 1 to Media 4 files. This series of images from a 40-frame movie were taken at 15 fps (time delays of ~0.065 between exposures) and are presented in the media files at 10 fps. These images have a magnification of 50× with a 660 nm source. Note that the cilia and their motion from frame to frame are highly visible as are small motions of the body and organelles within the body.

Fig. 7

Sample images of a time series movie of a large paramecium taken with 50× magnification at 660 nm. (a) Phase contrast (interference) (Media 1, movie showing interferogram images). (b) Simulated DIC (x gradient) (Media 2, movie showing simulated DIC). (c) Phase gradient magnitudes (simulated dark field) (Media 3, movie showing simulated dark field images). (d) Optical thickness (Media 4, movie showing optical thickness). Note motion of cilia in media files.

4.3. Removal of background surface structure

Ultimately, the objective of quantitative phase imaging is to follow motion or to find relative changes over time within a sample. In order to do this, we need to insure that we are isolating the object relative to the background so that the optical thickness corresponds to the object of interest and not some variation in the thickness of the coverslip, the shape of the reflective mirror under the sample, the thickness of the liquid layer, or alignment of the sample relative to the microscope. Normally when aligning an object for a measurement, we want to level the background as much as possible as shown in the examples of Fig. 6 and Fig. 7. But slight variations in the coverslip, mirror or liquid layer will not be eliminated this way and it is not always possible to remove the background shape by simply aligning the sample. Systematic errors in the optical system can be removed by well-known techniques of generating a reference surface containing errors within the optical system and subtracting that from data [34]. But this will not deal with variations within the object or its alignment. To do this, we need to differentiate between the object and the background.

To illustrate this, Fig. 8 shows cell cultures of the MCF715 human breast cancer line grown in cell media on coverslips. To image these cells, the coverslips are placed upside down on a highly reflective mirror with their growth media filling in between the mirror and coverslip. These images were taken at 20× with a 1.67× tube lens, a 660 nm source and 2 ms exposures. Figure 8(a) shows an interferogram with 4 fringes of tilt in the background. The fringes are slightly curved showing that there is also some curvature present due to the mirror or some misalignment. The unwrapped phase is shown in Fig. 8(b) while the raw optical thickness data are shown in Fig. 8(c). Typically in interferometer systems, tilt will be removed by a least squares fit of a plane to the data and then subtracting that plane from the data. When this is done on a relatively featureless scene, it works quite well, but when there are a number of objects within the field of view as in this example, it does not level the data very well relative to the background as seen in Fig. 8(d). When the background areas to fit the plane to are defined by the rectangles in Fig. 8(e), the resulting data are then independent of the background tilt as seen in Fig. 8(e). This manual technique works, but is tedious and will not work if the areas defined are encroached upon by moving objects as the entire time series is processed. Later in this paper we discuss processing techniques developed to do this processing automatically.

Fig. 8

Human breast cancer cells taken with 20× magnification at 660 nm. (a) Interference pattern showing 4 fringes of tilt. (b) Modulo 2π wrapped phase. (c) Unwrapped optical thickness in nm from phase. (d) Best-fit plane removed. (e) Optical thickness with manual background removal using the areas defined by the white rectangles (see text).

4.4. Dynamic studies of in vitro cell cultures

Figure 9 shows another human breast cancer cell culture taken at 20× after contact with various media to produce changes within the cells. All of these images are scaled in optical thickness to the same limits of 0 to 550 nm after the background has been removed so that changes are more obvious. In Fig. 9(a) the cells are in their growth media. In Fig. 9(b) the cells have been exposed to purified water causing them to osmotically swell. Figure 9(c) shows how they further swell and flatten after further exposure to purified water while Fig. 9(d) shows the cells after then being exposed to NaOH. For each of these cases, a times series of measurements were recorded every few seconds. Media 5 shows the entire time series.

Fig. 9

Time series of 3D phase images of another breast cancer cell culture taken at 20×. All images have the same optical thickness pseudocolor scale from 0 to 550 nm. (a) Cells in original media. (b) After contact with purified water the cells osmotically swell. (c) After more purified water the cells continue to swell and flatten. (d) After contact with NaOH the cells are beginning to break down. These movies were taken with sampling times of a few seconds over 23 minutes (Media 5).

After this series of images, the cell culture was then exposed to Alconox® (a detergent containing enzymes). The Alconox® further breaks down the cell walls and the cells dissolve. Figure 10 (Media 6) shows a series of measurements taken every 4 seconds as the cells were dissolving over a 78 second time span. Figure 10 shows snapshots every 10 seconds while Media 6 shows the entire movie. With more time resolution it is possible to watch how the cells break down and dissolve.

Fig. 10

Time series of phase images of breast cancer cells as they dissolve after contact with pure water, NaOH and Alconox®. Imaging area is 200 × 300 μm. Data are taken as 20× with a 660 nm source and 2 ms exposures. Samples above are every 10 seconds from a 78 second long movie (Media 6).

4.5. Simple background removal using histogram filtering

Because we are dealing with large numbers of images in a time series, there is a need to automate the background removal processing presented in Fig. 8. This can be implemented using many different algorithms, many of which are highly dependent on the method by which the data are collected and the type of scene being analyzed [38-40]. The first method we tested histograms the optical thickness data. When scenes contain small objects, have more than half the area in the background, and have histograms with distinct separate peaks for background and object, it is simple to filter the histogram to mask areas surrounding the object(s). Once the object(s) are masked out, a low-order Zernike surface can be fit in a least-squares manner corresponding to piston and tilt, and then subtracted to remove the background [4,41].

An example of this processing is illustrated in Fig. 11 where three consecutive frames from a 15 fps phase movie taken with 45× magnification (20× Linnik with a 2.25× FOV lens) at 660 nm of a moving paramecium are shown. The paramecium moves fast and relatively large distances per frame compared to its size, so manually identifying the background to remove for each frame is prohibitively time consuming. Tilt and subtle shape are noticeable in the top images where the left side of the images is higher than the right side (Figs. 11(a)–11(c) and Media 7). This is removed in the bottom set of images enabling the isolation of the object from the background (Fig. 11(d)–11(f) and Media 8). Another byproduct of this procedure is removal of the mean value from frame to frame. This effectively eliminates jumps between frames caused by noisy pixels (for example Fig. 11(b)), small pieces of debris in the cell solution or scratches on the mirror. However, since this technique assumes that the largest (and lowest in optical thickness) peak in the histogram is due to the background, the main limitation of this technique is that when many similar objects are present or if the object takes up more than half of the image area, the masking method does not always work.

Fig. 11

Three consecutive phase images of a swimming paramecium from a time series movie. (a)–(c) Raw unwrapped optical thickness from phase (Media 7). (d)–(f) After simple automated background leveling using histogram filtering (Media 8).

4.6. Automated background leveling using gradient thresholding

A more robust background-leveling algorithm has been developed that operates very well on scenes that have varied sizes of phase objects against a background. The algorithm is based on techniques that utilize thresholding with respect to gradients of the data [42,43]. By considering the gradient magnitudes of the phase, rather than the phase values, regions where the phase is slowly varying can be automatically identified. The general assumption is made that the slope magnitude value that has the maximum occurrence is assumed to be associated with the background. This assumption is generally valid for images obtained by this microscope. The background leveling routine consists of the following steps:

Calculate gradient magnitudes as defined in Eq. (10).

Iteratively threshold data outside a given range centered about the maximum occurrence of slope magnitude such that enough pixels have been masked.

Grow the mask to incorporate nearest neighbors of masked pixels.

Apply mask to phase data and calculate low-order Zernike background surface from the non-masked pixels.

Remove background surface from original phase data.

This technique is illustrated using an example where the object dominates the field of view and the simple method described in Sect. 4.5 does not work. The measured optical thickness data of the tail end of a rotifer are shown in Fig. 12(a) . The gradient magnitudes are calculated using Eq. (10) and shown in Fig. 12(e). Areas where the gradient magnitude is above a set relative threshold are masked out of the data set. For subsequent iterations, the gradient magnitudes of the remaining pixels are calculated, and pixels above a set relative threshold are further masked. This process is repeated until either the number of pixels masked reaches a set limit or until the values of the gradient magnitudes left in the data set are below a set relative threshold. Figure 12(f)-(i) show this process through 4 iterations. The final binary mask is shown in Fig. 12(b). Note that most of the pixels in the object have been masked out (white pixels), and variations in the background are also masked out. The code has not yet been optimized to minimize execution time. It currently takes a few minutes of post-processing to obtain a finished movie. This type of algorithm is compatible with implementation within GPU processors to provide real-time processing.

Fig. 12

Rotifer tail imaged with 50× at 660 nm. (a) Optical thickness calculated from raw phase (movie in Media 9). (b) Final binary mask corresponding to background area for fitting Zernike surface. (c) Best fit plane (synthetic 3-term Zernike surface) corresponding to background surface to remove. (d) Optical thickness after automatic background removal (movie in Media 10). (e) Gradient magnitudes from (a). (f)–(i) Iterations 1 to 4 to determine area to mask (see text).

Once this generated mask has been applied to the optical thickness data, the remaining data are fit with a Zernike surface using a least squares fit. Because the predominant background shape is a tilted plane, the fit is to the first 3 Zernike terms including piston (mean value) and tilt in the x and y directions (see Ref. [41] for a definition of these terms). The planar Zernike surface representing the background shape that needs removing is shown in Fig. 12(c). This surface is then subtracted from the original data (Fig. 12(a)) and shown in Fig. 12(d). At this point quantitative optical thickness data can be extracted from the object independent of the background. Remember that this assumes that the overall thickness of the coverslip, objects, and media are constant, and that there are not multiple objects stacked on top of one another. This processing enables relative changes from frame to frame in a time series to be quantified.

An example showing a higher-order term fit is shown in Fig. 13 . Figure 13(a) is a cropped subset of the data in Fig. 8. The dominating background shape is more complicated with a curved background requiring higher-order Zernike terms to fit a background surface. When larger-term Zernike fits are necessary, the need to ignore objects becomes more critical because the spherical shape of cells can confuse a simple background subtraction technique if those objects are not ignored with a mask. The gradient magnitudes are shown in Fig. 13(b) while the mask is shown in Fig. 13(c). A Zernike surface fit with 6 terms including piston, tilt, curvature, and cylinder is shown in Fig. 13(d). The optical thickness data with the background surface removed are shown in Fig. 13(e). Note that the resulting data have a smaller peak to valley and a more uniform height profile enabling better estimation of cell shapes and volumes than those obtained manually in Fig. 8. We would not expect to ever need a higher order fit than this to determine the background shape. Furthermore, this shape will remain consistent between frames of data taken in a time series as long as the system is thermally and mechanically stable.

Fig. 13

Automatic background removal from cropped portion of data set from Fig. 8. (a) Raw unwrapped optical thickness data. (b) Gradient magnitudes. (c) Binary mask after 4 iterations. (d) Zernike surface generated after fitting tilt, curvature and cylinder to unmasked points. (e) Optical thickness with background removed.

5. Discussion and Conclusions

This paper has described recent research into development of a dynamic quantitative phase imaging microscope and shown a number of examples of dynamic phase measurements of living biological organisms with an emphasis on automatic background leveling routines developed to process these data for further quantitative analysis. All data required to determine phase and optical thickness are gathered in a single snapshot utilizing a pixelated phase mask, so no scanning is necessary. Short exposure times freeze motion instantaneously. Data from brightfield imaging, phase contrast (interference image), as well as phase gradient magnitude and DIC images, are also obtained simultaneously along with phase and optical thickness.

Two automated background-leveling routines have been adapted and applied to a wide range of biological samples. By thresholding the gradient magnitudes of the optical thickness data, objects can be successfully masked, leaving only pixels associated with the background that can then be characterized with a low-order Zernike surface. This method successfully removes background shape without the need for user input and is easily scalable to process large numbers of data frames.

These methods can be extended to higher magnifications, immersion objectives, higher numerical apertures, a large range of wavelengths, and to measure cells in transmission. Harmless light levels offer a non-destructive means of observing and quantifying biological behavior and dynamic variations over time. The ability to dynamically measure biological organisms in real time opens up many different types of applications ranging from flow cytometry to tissue dynamics, morphological and volumetric studies along with mechanistic studies, process monitoring, quantification of cellular motion, monitoring and tracking cellular damage under known perturbations, tracking cell migration, nerve and muscle transmission, histology and photodynamic therapy. Conceptually, this model can be further modified to include simultaneous fluorescence measurements to more specifically track particular mechanisms.