Using Fourier ptychography microscopy to achieve high-resolution chromosome imaging: an initial evaluation.

SIGNIFICANCE: Searching analyzable metaphase chromosomes is a critical step for the diagnosis and treatment of leukemia patients, and the searching efficiency is limited by the difficulty that the conventional microscopic systems have in simultaneously achieving high resolution and a large field of view (FOV). However, this challenge can be addressed by Fourier ptychography microscopy (FPM) technology. AIM: The purpose of this study is to investigate the feasibility of utilizing FPM to reconstruct high-resolution chromosome images. APPROACH: An experimental FPM prototype, which was equipped with 4  ×    /  0.1  NA or 10  ×    /  0.25  NA objective lenses to achieve a theoretical equivalent NA of 0.48 and 0.63, respectively, was developed. Under these configurations, we first generated the system modulation transfer function (MTF) curves to assess the resolving power. Next, a group of analyzable metaphase chromosomes were imaged by the FPM system, which were acquired from the peripheral blood samples of the leukemia patients. The chromosome feature qualities were evaluated and compared with the results accomplished by the corresponding conventional microscopes.
RESULTS: The MTF curve results indicate that the resolving power of the 4  ×    /  0.1  NA FPM system is equivalent and comparable to the 20  ×    /  0.4  NA conventional microscope, whereas the performance of the 10  ×    /  0.25  NA FPM system is close to the 60  ×    /  0.95  NA conventional microscope. When imaging the chromosomes, the feature qualities of the 4  ×    /  0.1  NA FPM system are comparable to the results under the conventional 20  ×    /  0.4  NA lens, whereas the feature qualities of the 10  ×    /  0.25  NA FPM system are better than the conventional 60  ×    /  0.95  NA lens and comparable to the conventional 100  ×    /  1.25  NA lens.
CONCLUSIONS: This study initially verified that it is feasible to utilize FPM to develop a high-resolution and wide-field chromosome sample scanner.

Introduction

Leukemia is one category of blood cancers that commonly occur in both female and male patients. For the diagnosis and treatment of leukemia, chromosome karyotyping is critically important; it organizes and pairs all chromosomes in order of decreasing length. In this process, cytogeneticists have to search the whole specimen slide and identify the analyzable metaphase chromosomes, which is a time-intensive and tedious operation. To reduce the clinician’s workload and enhance the diagnostic efficiency, many research studies have been focused on the development of automated slide scanners and the associated computer-aided detection schemes,, aiming to accomplish quick cell digitization and identification. Despite some encouraging progress, the performance of the current scanners is majorly impeded by the long scanning time, which can be attributed to the limited space-bandwidth product of the objective lens., With few high-cost exceptions, this limitation indicates that it is very difficult for the objective lens to simultaneously accomplish a large field of view (FOV) and high spatial resolution, which implies that high-resolution objective lenses will have a much smaller FOV than the low-resolution lens. Considering that the chromosome images must be obtained with high spatial resolution to ensure enough details for clinical karyotyping, the corresponding high-magnification scanning will be significantly slower as each acquisition only covers a much smaller area of the specimen.

Meanwhile, the recent technology of Fourier ptychography microscopy (FPM), is an emerging strategy to address the challenge of limited space-bandwidth products. This method first collects a number of low-magnification images under different sample illumination conditions and then synthesizes them together to reconstruct a high-resolution image. Since the data acquisition is under low magnification, it inherits the advantage of low-resolution scanning systems, namely, the large FOV and large depth of field (DOF). For example, in current clinical practice, the objective lens is used to ensure the band pattern sharpness. Given the typical objective lens with an optical field number (OFN) of 22, the corresponding FOV is . Meanwhile, if FPM is utilized to achieve the same band pattern quality under the lens, the FOV of the objective lens (with the same OFN number) is enhanced to 2.2 mm; thus the scanning efficiency can be greatly enhanced. Moreover, due to the large system DOF, FPM can also vastly reduce the precision requirement of the moving stage, and the total cost of the scanner can be significantly reduced. Although this method has been applied in many different clinical scenarios, no previous research has focused on investigating the feasibility of utilizing FPM on high-resolution metaphase chromosome imaging.

In this study, we comprehensively assess the performance of an FPM system and its clinical utility on chromosome imaging. The modulation transfer function (MTF) curves are first estimated, aiming to assess the system resolving power. Next, the analyzable metaphase chromosomes acquired from the peripheral blood samples of leukemia patients are imaged by FPM system, and the chromosome feature qualities are compared with the results captured by the corresponding conventional microscopes. All experimental details are presented in the following sections.

Materials and Methods

Experimental Set-Up

In this experiment, a Fourier ptychography-based microscope was developed on the basis of the Thorlabs Cerna Microscope platform (Thorlabs, New Jersey, USA), which is equipped with either or Olympus objective lenses (Olympus, Tokyo, Japan) and a FL20BW CMOS camera (Tuscen, Fuzhou, China). The focal length of the platform tube lens is 150 mm; thus the actual magnification is and for the and objective lenses, respectively, as 180 mm is the standard focal length for the Olympus infinity corrected optical system. To accomplish the tilted illumination, a LED panel (Adafruit, New York, USA) was placed under the sample stage, with the vertical stage-sample distance being . During the image acquisition, the central 225 () green (4 mm spacing, central wavelength ) LEDs were used, which yields an illumination NA of . Therefore, the equivalent system is and for the and objective lens configurations, respectively. Our FPM system can also be performed as a regular microscope for imaging (0.40 NA). Meanwhile, the high-magnification comparative imaging was conducted on a Nikon Ni transmissive microscope (Nikon, Tokyo, Japan), which is equipped with or objective lenses.

Fourier Ptychography Theory and Image Reconstruction

FPM is essentially one kind of aperture synthesis technology., From the perspective of frequency domain, the objective lens is a low-pass circular filter. In regular imaging, the sample is illuminated along the perpendicular angle of incidence (Fig. 1, red). The objective lens works as a circular shape low-pass filter on the corresponding spectrum (i.e., its Fourier transform) , and the filter cutoff frequency is estimated as ( is the illumination wavelength). However, if the (partially) coherent light illuminates the sample with an angle of incidence in the direction (Fig. 1, blue), the corresponding spectrum will be shifted as , , and the objective lens will then filter the spectrum and collect the information within the same circle area, but the center is changed to . Therefore, if the sample is illuminated under a sequence of different angles of incidence, the system can collect the frequency information from different areas of the corresponding sample spectrum. After we computationally synthesize this information together, the cutoff frequency can be extended to , with being the maximal angle of incidence (Fig. 1, green circle). The resolving power will then be vastly improved.

Fig. 1

The principle of FPM technology.

Accordingly, reconstructing the high-resolution image from the acquired raw patterns is equivalent to reconstructing the corresponding spectrum. Given that the complex spectrum requires both intensity and phase information and the raw patterns only record the sample intensity, the missing phase information (phase retrieval) needs to be restored for the spectrum reconstruction. Moreover, the algorithm has to estimate the optimal pupil function to minimize the aberration and enhance the final image quality. As a result, in this study, we adopt a phase restoration algorithm embedded with pupil function recovery,, which iteratively updates the reconstructed spectrum until the preset accuracy tolerance is satisfied. For this method, we first divide the entire spectrum of the reconstructed image into different subregions, each of which corresponds to one obtained raw pattern. For each iteration, the spectrum update is performed subregion by subregion, which consists of the following three steps. First, pupil function is applied on the subregion spectrum : , where denotes the ’th iteration and denotes the ’th subregion. In addition, is defined as , and are the and direction illumination angles, respectively. The spatial representation is also computed by inverse Fourier transform correspondingly. Next, the captured raw pattern is applied on the spatial representation as the intensity constraint: . Finally, using Gaussian–Newton’s method, the subregion spectrum and pupil function are updated based on the constrained result .

Use Standard Resolution Target and Clinical Specimen to Evaluate the Resolving Power of FPM System

To assess the resolving power, the MTF curves of the FPM system were measured under two configurations (i.e., and raw image acquisition). As a comparison, we also measured the MTF curves of the conventional microscope equipped with , , and objective lenses. Currently, which approach should be the standard metric to assess the performance of a coherent system has not been determined as the object phase variation may enhance the system power of resolving two close points. Given that the standard resolution pattern does not contain any phase variance, our MTF-based performance assessment may actually underestimate the performance of the investigated FPM system. Meanwhile, it is widely accepted that the MTF curve can objectively evaluate (neither underestimate nor overestimate) the performance of incoherent imaging systems such as conventional microscopes. Therefore, when using the MTF curve, we would rather underestimate our FPM system than overestimate it, and the corresponding results of performance comparison still remain solid.

Accordingly, a high-resolution (up to ) USAF 1951 target (Newport, California, USA) was adopted, and the contrast of the imaged bar patterns was measured at a sequence of discrete frequencies from zero to the spatial resolution. During the measurement, the test target was placed on the sample stage and adjusted to the in-focused position. For each pattern on the captured target, the contrast was calculated by , where and are the average maximal and minimal values of the bar patterns, respectively. Based on these estimated contrast values, one smooth normalized MTF curve was generated by the curve fitting algorithm.

To evaluate its clinical utility, a certain number of analyzable metaphase chromosomes were also imaged by our FPM system. All of the cells were collected from the blood samples of leukemia patients and processed under standard clinical procedure in our medical center. The chromosomes were imaged by both the FPM system and the corresponding equivalent conventional microscopes (e.g., under and objective lenses). The chromosome band patterns depicted on each generated image were assessed and compared between these two groups of results (i.e., FPM versus conventional system). To evaluate the impact of the DOF on the feature qualities, each chromosome was imaged in a series of different positions, including the in-focused position and other positions outside the focal plane.

Results

Figure 2 demonstrates the reconstructed images of a 1951 USAF resolution target. In Fig. 2(a), the sample is directly imaged under the objective lens, which is able to resolve the patterns up to group 7-3 (). This result generally agrees with the theoretical computation (). As indicated in Fig. 2(b), when using FPM with the objective, the spatial resolution of the reconstructed image increases to (group 9-5), which is equivalent to the result directly captured under the objective lens [Fig. 2(c)]. Meanwhile, Figs. 3(a)–3(c) demonstrate the estimated MTF curves when utilizing the objective lens, FPM technology, and objective lens, respectively. As predicted by the Fourier optics theory, the normalized contrast value monotonically decreases as the spatial frequency increases. The curve of the objective lens slowly decreases from 0 to and significantly decreases from to the cutoff frequency (). Meanwhile, the contrast value of the FPM curve is above 0.7 when the frequency increases to , and the cutoff frequency is about . The MTF curve of the objective lens is similar to the FPM results, as demonstrated in Fig. 3(c).

Fig. 2

The images of a standard USAF 1951 resolution target, generated by (a)  objective lens; (b) FPM technology with objective lens; and (c)  objective lens.

Fig. 3

The measured MTF curves of different microscopes when using (a)  objective lens; (b) FPM technology with objective lens; and (c)  objective lens. Under these three imaging conditions, the cutoff frequencies are (a) , (b) , and (c) , and the contrast values decrease to 0.7 at (a) , (b) , and (c) , respectively.

Figure 4 demonstrates the microscopic images of one analyzable metaphase chromosome acquired from the blood samples of one leukemia patient. In clinical practice, the chromosome samples are stained by Giemsa dye to generate the “black and white” band (G-banding). Each chromosome has a unique band pattern as its own signature that is used for karyotyping or abnormality identification., The quality of the reconstructed images is largely determined from whether these band patterns are clearly resolved or not. Accordingly, when the cell is imaged under the objective lens [Fig. 4(a)], we can only distinguish this metaphase cell from other interphase cells with a circular shape but cannot recognize any details regarding the chromosome band patterns. Figures 4(c)/4(d) indicate the reconstructed amplitude and phase images by FPM equipped with the objective. Note that we rescaled the phase values to the range [0, 255] and then inverted the rescaled pixel values for figure demonstration because the raw phase image has a dark background (zero phase shift). A similar process was conducted on all of the following phase images. For the reconstructed amplitude/phase images [Figs. 4(c)/4(d)], some band patterns can be distinguished, and the band sharpness is comparable to the images captured by the objective lens [Fig. 4(e)].

Fig. 4

Sample images of one analyzable chromosome from the blood sample of leukemia patients, which are imaged using (a), (b)  objective lens; (c), (d) FPM technology with objective lens; and (e)  objective lens. (c) and (d) are the reconstructed amplitude and phase images, respectively.

When the raw patterns were captured under the objective lens, the reconstructed USAF 1951 resolution target is indicated as Fig. 5. Figure 5(a) shows the target image that was directly captured by the objective lens; the minimum resolvable bar pattern is group 9-1 (). When using the FPM technology [Fig. 5(b)], the spatial resolution vastly increases to (group 10-3), which is comparable to the result of the objective lens [Fig. 5(c)]. Under the objective lens [Fig. 5(d)], the target image resolves the pattern bar up to group 10-6 (), which is significantly higher than the FPM results. Meanwhile, the corresponding MTF curves of the objective lens [Fig. 6(a)] and FPM system [Fig. 6(b)] slowly decrease to 0.7 at a frequency of 381 and and then precipitously approach zero at the respective cutoff frequencies of 681 and . In contrast, the curves of the [Fig. 6(c)] and objective lenses [Fig. 6(d)] decrease rapidly first and then slowly attain zero at the frequencies of 1642 and , respectively.

Fig. 5

Images of the USAF1951 resolution target generated using (a)  objective lens; (b) FPM technology with objective lens; (c)  objective lens; and (d)  objective lens.

Fig. 6

The measured MTF curves of different microscopes when using (a)  objective lens; (b) FPM technology with objective lens; (c)  objective lens; and (d)  oil immersion objective lens. Under these four imaging conditions, the cutoff frequencies are (a) , (b) , (c) , and (d) , and the contrast values decrease to 0.7 at (a) , (b) , (c) , and (d) , respectively.

Next, Fig. 7 shows the same analyzable chromosome imaged using FPM technology and different objective lenses. For the result of the objective lens [Fig. 7(a)], the chromosome bands can be generally recognized, but the pattern details are not recognizable. When using the FPM technology with the objective [Fig. 7(b)], the quality of the band patterns is vastly enhanced, so the bands are analyzable for the subsequent karyotyping. The reconstructed band patterns are somewhat better than the results of the objective lens [Fig. 7(d)] but slightly inferior to the results of the objective lens [Fig. 7(e)].

Fig. 7

The images of a sample chromosome acquired from the patient’s blood sample, which were captured or reconstructed using (a)  objective lens; (b), (c) FPM technology with objective lens; (d)  objective lens; and (e)  oil immersion objective lens. (b) and (c) are the reconstructed amplitude and phase images, respectively.

Finally, Figs. 8 and 9 indicate the reconstructed images that were captured at different focusing positions. For Fig. 8(a), the raw patterns used for reconstruction were acquired under the objective lens at the in-focused position, and the raw patterns of Fig. 8(b) were captured away from the in-focused position. Compared with Fig. 8(a), the band patterns in Fig. 8(b) are somewhat deteriorated but still acceptable. Meanwhile, when the chromosomes were directly captured under the objective lens [Figs. 8(c), 8(d)], the band patterns become somewhat deteriorated when it was placed only away from the in-focused position. Similarly, the raw patterns used for the reconstruction of Figs. 9(a) and 9(b) were captured under the objective lens, which were at in-focused and away positions, respectively. The band patterns in Fig. 9(b) are generally as sharp as the band patterns in Fig. 9(a). However, when imaging under the objective lens [Figs. 9(c) and 9(d)], the chromosome bands become fuzzy when the sample is place away from the focal plane.

Fig. 8

The images of a sample chromosome obtained from the patient’s blood sample. The raw patterns of the chromosome were captured under the objective lens and reconstructed by FPM technology. The chromosome was placed at the (a) in-focused position and (b)  away from the in-focused position. The chromosome was captured directly by the objective lens, which was placed at the (c) in-focused position and (d)  away from the in-focused position.

Fig. 9

The images of a sample chromosome obtained from the patient’s blood sample. The raw patterns of the chromosome were captured under the objective lens and reconstructed by FPM technology. The chromosome was placed at the (a) in-focused position and (b)  away from the in-focused position. The chromosome was captured directly by the objective lens, which was placed at the (c) in-focused position and (d)  away from the in-focused position.

Discussion

In this study, we applied FPM technology on chromosome imaging and thoroughly investigated the band pattern quality depicted on the reconstructed images. To the best of our knowledge, this is the first study that utilizes FPM technology on chromosome imaging, and it has two unique characteristics.

First, although the equivalent NA of our FPM system (raw patterns under the objective lens) is far , the band quality of the reconstructed images is close to or even comparable to the results captured by the objective lens. In the previous research,, it was cautioned that the direct comparison between the equivalent NA of FPM and the native NA of conventional microscopy may not be reasonable. However, our FPM system may better resolve the chromosome samples containing phase information. Meanwhile, we believe this phenomenon can be attributed to two other factors as follows. First, the effective NA of the objective lens is actually higher than its rated value (0.25). As indicated in Fig. 5(a2), the bar patterns are resolvable up to group 9-1, with a half-pitch resolution of . Accordingly, the computed NA is , which is higher than 0.25. In this research, the NA of illumination is configured as 0.38; thus the final equivalent NA should be 0.71. This value generally agrees with the measured results [ 0.84, Fig. 5(b)]., Second, the MTF curve is significantly different between the FPM system and equivalent conventional microscope. The FPM curve decreases slowly first and then sharply approaches zero; but the curve of conventional microscope decreases sharply first and then slowly approaches zero. As a result, at a very wide range of frequency (0 to ), the FPM microscope can provide better resolving power (i.e., contrast) than the and conventional objective lenses, although these two lenses have higher cutoff frequencies. Meanwhile, most chromosome band patterns are bright-dark alternating strips with a spatial frequency of , and the FPM system may provide better contrast on these bands, which explains the comparative quality of the band patterns on the reconstructed images. When the raw patterns were captured under objective lens, the MTF curve of the FPM is similar to the equivalent objective lens; thus the band pattern qualities are comparable on the images generated by these two systems.

Second, this investigation points out a new strategy to accomplish efficient slide scanning for chromosome karyotyping. For the microscopic chromosome image, we demonstrated that the FPM system can achieve a comparable result to that of the conventional objective lens. At the same time, the major advantages of the lens still remain: large FOV and large DOF. This discovery provides a new possible strategy for achieving high-speed chromosome digitization: acquire the raw images under the objective lens and then reconstruct the high-resolution analyzable metaphase chromosomes for the following diagnosis work. In addition, we also demonstrated that the reconstructed images of the raw patterns achieves a similar quality as compared with the objective lens, which is adequate for identifying the analyzable cells. Then the efficiency may be further enhanced if the slide is captured and reconstructed first under for the initial screening and then only the selected regions of interest are imaged again under the condition to synthesize high-resolution images. By adjusting the focal length of the tube lens or directly using the available lower magnification objective lens (i.e., or ), it is also possible to further enlarge the FOV to achieve an even higher scanning speed without any quality degeneration of the final images.

Although the results are encouraging, we recognize that this study has a number of limitations. First, we only used the analyzable metaphase chromosomes acquired from blood samples for feature quality analysis, which is representative but not comprehensive. More types of clinical samples should be investigated, including bone marrow, amniotic fluid, and product of conception. Second, we did not apply the adaptive wave correct technology to further enlarge the system DOF. Third, there is still a large potential to enhance the equivalent NA of the FPM system. In this study, the raw patterns were obtained only under and objective lenses, and the illumination NA was only achieved as 0.38. The high-resolving power objective lens (i.e., ) can be used, and the illumination NA can be enhanced to 0.7 by adjusting the LED-sample distance and reconfiguring the illumination pattern on the LED matrix. Fourth, only the conventional optimization-based reconstruction schemes were adopted in this investigation. The deep learning-based data-driven algorithm is an emerging trend for phase retrieval and pupil function recovery, but they were not used in this research. Despite these limitations, we believe that this study is highly valuable for the development of a high-throughput chromosome sample digitizer in the future.