Physical pupil manipulation for speckle reduction in digital holographic microscopy.

The reduction of speckle noise by physically changing the pupil of the imaging system, as first envisioned in optical holography, is experimentally applied to a digital holographic microscope (DHM). The imaging pupil of a DHM, operating in image plane telecentric-afocal architecture, is changed in a controlled way between successive recordings, allowing the shooting of multiple partially-decorrelated holograms. Averaging the numerically reconstructed holograms yields amplitude and/or phase images with reduced speckle noise. Experimental results of biological specimens and a phase-only resolution test show the feasibility to recover micron-sized features in images with reduced speckle noise.

Introduction

Speckle noise, the granular pattern that arises from coherently-illuminated objects, is an unavoidable condition inherited from the roughness that most real-world materials have in the optical scale [1]. Although this phenomenon is widely used in metrological applications [2], in coherent imaging it is considered an annoyance that must be reduced or eliminated [3]. Holography is one field of application where speckle effects are especially deleterious. The suppression of speckle noise in holographic techniques is a particularly difficult problem due to their dependence on the light coherence to produce the images [4]; in consequence, multiple methods have been envisioned since the very onset of the field to reduce its incidence, and it continues to be a widely active field of research. In particular, in digital holography (DH), the problem has been lengthily studied; Ref. [5, 6, 7] present comprehensive reviews of the available methods, both optical and numerical, to deal with speckle noise.

Among the optical methods to reduce the effects of speckle noise in holographic techniques, the most common approach is averaging multiple images of the same scene that have been acquired with different random noises [4]. In these methods, known as multi-look approaches, the multiple realizations of the noise can be obtained by an ample range of procedures that include the use of a rotating diffuser [8, 9, 10], changing the wavelength [11] or the polarization state [12] of the illumination, and using different angles of recording [13, 14]. Most of these techniques have been successfully applied both on optical and digital holography, either in the recording or the reconstruction stage.

In this work, an optical multi-look method in which the multiple speckle fields are obtained by physically altering the pupil function of the imaging system is implemented in Digital Holographic Microscopy (DHM). The manipulation of the imaging pupil is a denoising strategy that has been widely reported in optical holography [15, 16, 17, 18, 19, 20, 21], and also numerically implemented in its digital counterpart [22, 23]; however, the existing literature has been mostly interested in the denoising of intensity reconstructions. The digital world has given holography the ability to directly access the phase information of the samples, enabling the development of label-free quantitative phase imaging (QPI) techniques [24]; DHM directly benefits from this capability as it allows an ample range of applications, for instance in the biological sciences, where most samples are both of micrometric dimensions and translucent [25], thus encoding their parameters in the phase information. Therefore, in this work, the denoising method by pupil manipulation is applied in the observation of biological samples and a phase-only resolution test, validating its feasibility to recover micron-sized features of translucent samples with enhanced contrast in numerically reconstructed phase maps, and also showing the validity of the optically-inspired methods to reduce the speckle noise in the digital-based techniques. For guiding the reader about the performance of the present method in comparison with other multi-look denoising approaches, the imaged biological samples are also denoised by means of the rotating diffuser method; some conclusions about the comparative performance are also stated.

Speckle reduction by pupil control in optical holography

The idea of reducing speckle noise in image-plane holograms by introducing moving pupils into the experimental setup was first proposed by Dainty and Welford in 1971 [15]. Their work, applied in the reconstruction stage of an optical hologram, shows that an aperture, smaller than the pupil, moving rapidly in the pupil plane of the reconstructing objective can reduce the speckle in the image plane, at the expense of the resolution and the available light. Figure 1 shows the experimental setup required for such a proposal.

Figure 1

Speckle reduction by moving aperture in the reconstruction stage of plane-image optical holography. Based on the setup diagram in [15].

The process of physically moving the aperture in the pupil plane, and then integrating the resulting intensity over time, was then understood as a filtering process over the spatial frequencies of the object; as such, its denoising effect was mathematically and experimentally described by Hariharan and Hegedus in terms of the power spectrum of the associated speckle patterns [17]. The following mathematical description is a summary of their results; to avoid unnecessary work duplication, the reader is referred to the corresponding reference for a complete treatment of the problem.

As illustrated in Figure 2, a circular aperture, smaller than the pupil of the imaging system, is assumed to be displaced in the pupil plane between successive exposures. If N exposures are done with different positions of the aperture, the resulting irradiance will be the sum of the corresponding contributions , where the impinging field at the i-th position of the aperture in the pupil planeis given by the convolution between the reconstructed object information, , and the impulse response of the system for that position of the aperture, .

Figure 2

Relation between the fixed and moving pupil for multi-look speckle reduction in optical holography. Based on the diagram in [15].

It was shown that the average power spectrum of the image irradiance is given by the sum of the power spectrum of a spatially incoherent object, having the same radiance as the noise-free object information, and the speckle power spectrum. The latter is shown to be given, for the case under consideration, bywhere and are the transfer functions of the system in the i-th and j-th positions, respectively, as given by the aperture, and is the autocorrelation of the noise-free object information and thus constant. If, for convenience, only two exposures are considered, is assumed to be unitary, and the pupil is taken to be aberration-free, Eq. (2) becomeswith being the displacement between the positions of the aperture in each exposure. Therefore, for a given displacement of the aperture, the speckle power spectrum would be determined by the sum of the incoherent transfer function of the system with an aperture two times: one for a spatial frequency , as given by the first term to the right in Eq. (3), and one for a spatial frequency , as given by the second term to the right in Eq. (3).

Within this framework, one can envision the reduction of speckle noise by superimposing reconstructions with varied speckle wavefields produced by a variety of pupil functions with different sizes, shapes, and positions. This observation led to further implementations with different geometries and movements of the filter mask [18, 19, 20, 21]. As will be presented in the next section, the experimental conditions of DHM present a direct parallel with these results, allowing the pursual of an implementation of this speckle reduction method to be sought.

Speckle reduction by pupil control in DHM

DHM can be implemented with multiple architectures. In the case of translucent samples, the most common one corresponds to a transmission DHM setup, as illustrated in Figure 3. In this setup, a digital camera, usually a charge-coupled device (CCD) or complementary metal-oxide semiconductor (CMOS), records the amplitude superposition of a reference and an object wave , which produces a steady interference pattern if both waves are generated from the same coherent optical light source, like a laser. is set to travel directly to the digital camera, while propagates through the sample, gathering its information. The object wave is, thus, the propagation of the complex-valued wavefield at the sample plane through the imaging system composed of the microscope objective (MO) and the tube lens (TL).

Figure 3

Configuration diagram of a Digital Holographic Microscope operating in transmission mode.

To ensure that any curvature-phase perturbation is removed from , and to guarantee the obtention of trustable QPI maps, the MO and TL must be configured in a telecentric-afocal architecture [26, 27, 28, 29]. The fulfillment of this condition allows the complex-valued object wavefield at the recording plane to be written aswith being the overall magnification of the microscope given by the focal lengths of the TL and the MO, respectively, denoting the 2D convolution operation, and the illumination wavelength. In Eq. (4), is the diffraction pattern of the pupil function of the microscope at the digital camera plane:

The reconstruction of the complex-object wavefield from the hologram can be done by following any interferometry method [30]. In particular, for the afocal-telecentric configuration operating at diffraction limit [28] utilized in this work, the retrieval of is done directly by numerical spatial filtering [31, 32] with no further propagation; allowing the obtention of a non-distorted wavefield like that in Eq. (4). This means that all the features of the digitally-recovered are expressed in the just said equation, in direct analogy to the optical holography case of an image-plane hologram, like the one used in Ref. [15]. The speckle noise, inherited from the use of coherent illumination to gather the sample information [1] must also be present in. In Eq. (4), is the geometrical image of the sample because the sample plane and the camera plane are conjugated; therefore, all the diffractive effects, including the speckle noise, are left to the point spread function in Eq. (5) [33]. Just like in optical holography, the light spots that compose the geometrical image of have their size, position and shape controlled, likewise, by the size, position and shape of the pupil function . Under this understanding of the imaging process, it is reasonable to expect that the results from optical holography can be directly applied to the experimental conditions of DHM by physically modifying the imaging pupil in the object arm during the recording stage.

Experimental setup

A transmission DHM, operating in an afocal-telecentric configuration and the diffraction limit, is set up as illustrated in Figure 3. If the effective focal length of the MO is not enough for the optomechanical components of the moving aperture to be properly positioned at its pupil plane, a relay lens can be used to project the pupil plane further away from the objective. The physical manipulation of the aperture can be easily achieved with off-the-shelf components by using, for instance, an iris diaphragm mounted into an X–Y displacer, giving control over both the size and position; however, if the use of customized components is possible, faster experimental results can be obtained by using a rotation mount coupled to either a linear displacer with an iris diaphragm or a pair of pupils, like those of Ref. [15] illustrated in Figure 2. With the setup completed, the aperture can be freely manipulated while registering a hologram for each different realization. Once all the desired digital exposures are completed, their numerical processing follows the same process as any other multi-look denoising methodology; namely, each of the holograms is numerically reconstructed for the desired information, either phase or intensity, and latter averaged to finally obtain a denoised image. Figure 4 presents a summary of this approach.

Figure 4

Flowchart of the method for speckle noise reduction by physical manipulation of the pupil function.

For the experiments in this work, the microscope was configured using an illumination wavelength of 533 nm and a CMOS camera with 1024 × 1024 square pixels with a side length of 5.2 μm. The object arm was composed of an infinity-corrected 10x/0.25 MO, and a TL with 200 mm of focal length; the changing aperture was created using an iris diaphragm, larger than the system's pupil size, mounted on an X–Y displacement cage. This configuration, located at the pupil plane of the MO as shown in Figure 5, allowed the manipulation of the aperture size from its fully open configuration to 20% of the pupil size, while keeping the possibility of scanning the entire original pupil. Examples of the different positions that the modified aperture can take are shown in panels (b), (c), and (d) of Figure 5 for an aperture whose size has been reduced to 80%, 50%, and 30% of the original pupil's diameter, respectively, and displaced in a circular path concentric to the system's pupil.

Figure 5

Movements and sizes of the aperture used for speckle noise reduction. (a) The moving aperture is located at the pupil plane of the object arm of the DHM. (b/c/d) Movement path of the aperture with 0.8/0.5/0.3 times the original pupil diameter.

In these displacement paths, the different realizations of speckle are expected to be only partially uncorrelated due to the overlap in the aperture positions. As most denoising methods based on multiple exposures require the superposed fields to be uncorrelated [7], the expected performance of the method is intrinsically related to how much of a decorrelation can be introduced with the selected degrees of freedom. Therefore, to predict the degree of correlation for the aforementioned configuration, the system was numerically simulated using a realistic representation of the speckle noise for the experimental conditions of DHM [34]; the sample was assumed to be a diffuser with superficial roughness in the order of λ/4, imaged with multiple aperture positions and sizes. The correlation between a pair of consecutive speckle intensity realizations, denoted A and B with respective means and , was measured using the discrete form of the Pearson's correlation coefficient [35].

The results are summarized in Figure 6 for the two available degrees of freedom. Panel (a) shows the variation in the correlation coefficient against the displacement of the aperture, with the latter being measured in its radius units. Three aperture positions are shown in panel (b), taken with displacements of 0.5, 0.6, and 0.95 times the aperture radius, for the red (dashes), green (dots), and blue (dash-and-dot) lines, in that order. In this same panel, the white circle represents the original pupil of the system, spatially centered in the pupil plane, and the crosses indicate the center position for each pupil displacement.

Figure 6

Correlation factor between consecutive speckle realizations. (a) Change of the correlation coefficient against the aperture displacement from the center measured in its radius units. (b) Illustration of the three different displacements highlighted in panel (a). (c) Correlation coefficient change against the aperture radius reduction percentage. (d) Illustration of the three different radii highlighted in panel (c). The color and dashing coding are shared between each pair of panels.

Panel (c) shows the variation in the correlation coefficient against the percentual aperture radius reduction. Panel (d) shows the three reduction cases of Figure 5; namely, 80%, 50%, and 30% of the original pupil radius for the red (dashes), green (dots), and blue (dash-and-dot) lines, respectively. In this panel, the white circle represents the original pupil of the system and the yellow cross indicates the spatial center of the pupil plane.

The behavior seen in Figure 6 is consistent with the expectation of only partially uncorrelated speckle fields for the illustrated displacements. This condition limits the method's efficiency if only small modifications to the pupil are considered. Nonetheless, it was shown in Ref. [17] that, under such conditions of low-decorrelation, a noise-reduction effect is still to be found in the object intensity; therefore, it is reasonable to expect that a similar filtering effect holds for the phase information. This was corroborated with a phase-only resolution test and later applied to a biological specimen.

Experimental results

The experimental setup described in the previous section was used to image a calibrated star test resolution target, over which the described speckle noise reduction method was applied. As the target was made of acrylate polymer on glass, allowing it to be considered as a pure phase object, all the resulting images are quantitative phase maps. For each aperture configuration, the improvement of the image quality was measured using the well-known metric of speckle contrast [4]; it was computed over a background region between the spokes of the target as , where is the standard deviation and the average of the phase values in the selected region. Additionally, according to the manufacturer's information, the used target has a 400 μm external diameter and 40 identical spokes; therefore, by measuring the minimum resolvable circumference in the denoised phase map, the resulting resolution power (RP) can be found. From these two values, the resolution penalty associated with the application of the proposed method can be measured and compared to the denoising performance for different aperture configurations. To ease the comparison, both the speckle contrast and the resolution power measurements were normalized against the original noisy image, such that their initial values are unitary.

Initially, the performance of the method when the imaging pupil is only modified displacing the aperture while its original size is kept was evaluated. Figure 7 shows the behavior of the two aforementioned metrics for this case, where each point represents the measurements over the averaged phase map from 8 different positions of the aperture, following the paths illustrated in Figure 5. As is expected from the partial correlation of the speckle fields predicted in Figure 6, the speckle contrast steadily decreases as the displacement of the aperture increases. This behavior can be seen in the dot-marked dashed-line series of the graph. The resolution power, represented by the diamond-marked solid-line series in Figure 7, remains essentially unaffected by the increasing alteration of the pupil function; thus, one can seek the maximum achievable improvement in the image quality without an evident resolution loss. Such condition is attained when the aperture is displaced 0.95 times the pupil radius (rp), producing a reduction of the speckle contrast by approximately 25%.

Figure 7

Relation between speckle-noise reduction with the proposed method and the resulting spatial resolution for multiple displacements of the aperture. The diamond-marked-solid-line series shows the resolution power in the denoised image, while the dot-marked dashed-line series shows the speckle contrast measured in a background area. Both curves are measured following the paths indicated in Figure 5 after averaging 8 phase reconstructions with a given displacement in units of the original pupil radius. The values are normalized against the corresponding measurements of the original phase map.

It is possible to further enhance the denoising effect of the method by introducing the size changes as a second freedom degree into the pupil's manipulation. To evaluate the performance of this case, the two metrics were measured over the averaged phase map resulting from 8 different positions of the aperture with a reduced size and displaced 0.95 times its radius, once again following the paths illustrated in Figure 5. The results are summarized in Figure 8 using the same conventions as the previous figure. As predicted by the correlation measurements in Figure 6, a further reduction in the aperture size allows the obtention of a more decorrelated speckle field, which in turn leads to a higher degree of noise reduction when the average is taken. Unlike the previous case, however, the resolution power is affected. This is expected, as the reduction in aperture diameter must be associated with a rejection of higher-order spatial frequencies during the recording stage. For the small size reductions, the effect is partially compensated by the scanning of multiple positions of the aperture, whose superposition behaves as the effective imaging pupil, although it must be noted that it will never be larger than the original pupil size. Therefore, a trade-off must be sought between the degree of denoising and the required resolution when the method is applied.

Figure 8

Relation between speckle-noise reduction with the proposed method and the resulting spatial resolution for multiple sizes of the aperture. The diamond-marked solid-line series shows the resolution power in the denoised image, while the dot-marked dashed-line series shows the speckle contrast measured in a background area. Both curves are measured following the paths indicated in Figure 5 after averaging 8 phase reconstructions with a given aperture radius, displaced 0.95 times the resulting radius. The values are normalized against the corresponding measurements of the original phase map.

When considering a displacement of 0.95 times the aperture radius, a maximum reduction of 30% of the pupil size can be applied without significantly degrading the resolution; this limit configuration allowed a total reduction of the speckle contrast to 0.7 times its original value. Above this limit, the scanning is not enough to fully compensate for the higher-order spatial frequency rejections, and thus the resolution is inevitably affected. Nonetheless, if the application allows the resolution penalty to be taken, further decreasing the aperture radius may yield speckle contrast reductions of up to 65% with only 8 exposures.

Finally, to visualize the results of the above-studied behavior, Figure 9 illustrates 3 denoising cases. Panel (a) shows a 3D rendering of the initial noisy phase map obtained with the original pupil of the system. Panels (b), (c), and (d), show the renders of the averaged phase maps obtained after the aperture was manipulated using the sizes and movement paths previously illustrated in Figure 5, taking 8 different exposures in each case. As before, the speckle contrast (C) of each averaged phase map is taken over the background region highlighted between the spokes of the target, and the resulting resolution power (RP) is measured from the minimum resolvable circumference; additionally, both values were normalized against those of panel (a) to ease the comparison.

Figure 9

Speckle denoising by physically moving an aperture at the imaging pupil. (a) Noisy phase map obtained with a fully open and centered pupil. (b/c/d) Denoised image after averaging the reconstruction of 8 different displacements of an aperture with 0.8/0.5/0.3 times the original pupil diameter. The speckle contrast is measured in the yellow-bounded region. The close-ups on the red-encircled regions attest to the loss in resolution. The color scale bar applies to phase values in all panels. RP is the resolution power and C the speckle contrast, both normalized against the corresponding values of panel (a).

The original noisy phase map, in panel (a) of Figure 9, yields a maximum resolution of 1.26 μm. This value, which is consistent with the theoretical expectation for the 10x/0.25 microscope objective employed, is also obtained for panel (b) of the same figure when the aperture is reduced to 80% of its original size. In this latter panel, the average achieved a reduction of 29% in the speckle contrast. In panel (c), where the aperture diameter was only half of the original pupil size, a speckle contrast reduction of 48% was achieved, while the effective resolution decreased to 1.46 μm. Finally, when the aperture had a diameter of only 30% of the original pupil's, as shown in panel (d), a total reduction in the speckle contrast of 60% was reached, but the resolution dropped to 2.39 μm. The accompanying insets of each panel show a close-up on the central region of the test, where the inner dashed circle marks the resolution limit for the corresponding reconstruction. The penalty on the final resolution of the system when the described method is taken to its limits can be directly seen by comparing these panels.

While the application of the presented method in the resolution target in Figure 9 shows great promise in the desired noise reduction, this sample has a relatively simple morphology and, due to its benchmarking nature, a highly controlled roughness; in consequence, having verified the feasibility of the denoising capabilities of physically controlling the imaging pupil in quantitative phase images, and having identified its resolution trade-off, the methodology is applied to a biological specimen: a thin section taken from the head of a Drosophila Melanogaster fly. This sample has intricate structures present in the interior of its features, thus representing a challenging application case.

The denoising results are shown in Figure 10; panel (a) presents the phase map acquired with the original pupil of the system fully opened, and panel (b) shows the final averaged phase image resulting from 8 different positions of an aperture reduced to 0.4 times its original diameter. If the speckle contrast inside the rectangle-bounded region in panel (a) is assumed to be unitary, the denoising method achieves a total reduction of 65% in the speckle contrast when measured in that same region in panel (b). This improvement can be further seen in the close-up circular insets, where the finer details of the sample acquire increased visibility.

Figure 10

Speckle denoising of a thin section from the head of a Drosophila Melanogaster fly. (a) Phase reconstruction obtained without modifying the system's pupil. (b) Image with reduced speckle noise after averaging the reconstructed fields from 8 different displacements of an aperture reduced to 40% of the original pupil radius. The circle-bounded region is three times magnified. The speckle contrast is measured in the rectangle-bounded region. The color scale bar applies to phase values in both panels..

This result was finally compared to the well-known multi-look speckle denoising technique of using a rotating diffuser [10]; the comparison is summarized in Figure 11, with panel (a) showing the same denoising result of Figure 10, and panel (b) the result of using the rotating diffuser. Overall, a similar improvement is noticed in both approaches; the region bounded by the yellow square, which is magnified in the lower left inset in both panels, shows that the background information reaches a comparable smoothness, with a further reduction of the speckle contrast for the method proposed in this work. However, the region bounded by the red circle, magnified in the lower right inset of both panels, shows again the trade-off of the described technique: due to the reduction in the effective aperture size, and consequent rejection of higher-order spatial frequencies, the internal structure of the sample loses sharpness, thus reducing the contrast of its smallest features. Because the multi-look method based on the rotating diffuser preserves the size of the imaging pupil, the above-mentioned spatial resolution trade-off does not occur.

Figure 11

Speckle denoising in phase map by averaging multiple recordings obtained by (a) physically varying the system's pupil, and (b) rotating a diffuser. The square-bounded region shows an equally denoised background region. The circle-bounded region shows better preservation of higher spatial frequencies in the result of panel (b).

Both the resolution test and biological sample results show that the described method of physically modifying the pupil function is a viable alternative to reduce speckle noise in phase images obtained from DHM architectures. Furthermore, despite the resolution trade-off, the experimental result with the biological sample shows that the described methodology is even suitable for complex internal structures, whose visualization is prone to be highly affected by the deleterious effects of speckle noise.

Conclusion

By understanding the coherent imaging system in the object arm of a Digital Holographic Microscope (DHM) as linear in amplitude, all its diffraction-derived effects, like speckle noise, can be directly attributed to the impulse response of the system. A denoising approach for the recovered object wavefield, which can be described as the convolution between this impulse response and a geometrical image of the sample, can thus be sought by manipulating the imaging pupil to obtain multiple partially uncorrelated fields to be statistically averaged. This denoising methodology, initially envisioned for the reconstruction stage of optical holography, was implemented in the object arm of a DHM operating in a telecentric-afocal architecture during the recording stage. With the controlled variation in size and displacement of an aperture in the pupil plane of the microscope objective, whose correlation degree was numerically measured, phase reconstructions with enhanced contrast and reduced noise levels were achieved.

The method was successfully applied to a phase-only resolution test target and a thin section from the head of a Drosophila Melanogaster fly. The former allowed the visualization of the resolution trade-off associated with the described method due to the reduction in the effective aperture of the system, while the latter showed promising results in the improvement of the visualization of micron-sized details in quantitative phase images (QPI).

The biological sample result was then compared with the denoising achieved via a rotating diffuser, showing similar outcomes. This comparison highlighted the possibility of applying optically-envisioned multi-look methods in the digital realm, which eases the data manipulation as the multiple configurations are not limited to the exposure time of the analog recording medium, and allow each exposure to be handled separately. Both methods may allow obtaining similar denoising results; however, due to the only-partial decorrelation introduced by the pupil approach, the configurations that allow the same improvement to be achieved involve a penalty in the effective resolution. To serve as a guide to the reader at the time of deciding which method to implement according to the particular application needs, this difference in the performance of both methods, particularly regarding the trade-off between the spatial resolution and the speckle reduction, is illustrated for the different configurations of the moving aperture.

These results show that the physical control of the pupil in the object arm of a DHM is worth considering among the set of tools available to researchers for reducing the deleterious effects that speckle noise introduces in QPI recovered in this technique.

Declarations

Author contribution statement

Jorge Garcia-Sucerquia: Conceived and designed the experiments; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Carlos Buitrago-Duque: Performed the experiments; Analyzed and interpreted the data; Wrote the paper.

Funding statement

Jorge Garcia-Sucerquia was supported by the Universidad Nacional de Colombia (HERMES 50210).

Data availability statement

Data will be made available on request.

Declaration of interests statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.