Literature DB >> 35960754

Compressed fluorescence lifetime imaging via combined TV-based and deep priors.

Chao Ji1,2, Xing Wang1,2, Kai He1, Yanhua Xue1, Yahui Li1, Liwei Xin1, Wei Zhao1,2,3, Jinshou Tian1,2, Liang Sheng4.   

Abstract

Compressed fluorescence lifetime imaging (Compressed-FLIM) is a novel Snapshot compressive imaging (SCI) method for single-shot widefield FLIM. This approach has the advantages of high temporal resolution and deep frame sequences, allowing for the analysis of FLIM signals that follow complex decay models. However, the precision of Compressed-FLIM is limited by reconstruction algorithms. To improve the reconstruction accuracy of Compressed-FLIM in dealing with large-scale FLIM problem, we developed a more effective combined prior model 3DTGp V_net, based on the Plug and Play (PnP) framework. Extensive numerical simulations indicate the proposed method eliminates reconstruction artifacts caused by the Deep denoiser networks. Moreover, it improves the reconstructed accuracy by around 4dB (peak signal-to-noise ratio; PSNR) over the state-of-the-art TV+FFDNet in test data sets. We conducted the single-shot FLIM experiment with different Rhodamine reagents and the results show that in practice, the proposed algorithm has promising reconstruction performance and more negligible lifetime bias.

Entities:  

Mesh：

Year:  2022        PMID： 35960754      PMCID： PMC9374265          DOI： 10.1371/journal.pone.0271441

Source DB:  PubMed          Journal:  PLoS One        ISSN： 1932-6203            Impact factor:   3.752


1. Introduction

Widefield fluorescence lifetime imaging (FLIM) is widely used in biomedical diagnostics and flow quantitative measurements, such as cancer diagnosis and treatment monitoring [1, 2], identifying species concentration from reactive-flow systems [3], and understanding the transient evolutionary behavior of eddies in highly turbulent flames [4]. Most of these examples are non-repeatable transient events that demand a single-shot widefield measurement method. However, performing high precision widefield lifetime measurements and quantitative analyses have always been a significant challenge in this field. The traditional widefield FLIM approaches, including time-correlated single-photon counting (TCSPC) [5, 6], streak camera [7], and single-photon avalanche diode (SPAD) [8, 9] possess high temporal resolution. Nevertheless, they require repeated measurements to obtain the widefield fluorescence lifetime. Recently, a snapshot compressive imaging (SCI) method, compressed ultrafast photography (CUP), has emerged as a potential solution for snapshot widefield FLIM [10]. Compared to traditional methods, CUP is the only passive 2D technology with picosecond to femtosecond time resolution, which can acquire complete 2D transient processes within a snapshot. The CUP system is a combination of the streak camera and compressive sensing methods. The typical CUP process is to map 3D encoded data onto a 2D detection array, and then restore the original information through compressed sensing algorithms. However, the data reconstruction step of CUP is a complex task. Significantly, the reconstruction quality of the image deteriorates rapidly with increasing sequence depth. To solve this issue, numerous algorithms have been designed through the exploration of underlying sparsity structures. Plug and Play(PnP) [11] is a typical SCI framework that allows the matching of flexible state-of-the-art forward models with advanced priors or denoising models. On this basis, GAP-TV has become a popular low memory and fast SCI algorithm that combines generalized alternating projection (GAP) and Total Variation (TV) [12]. Denoiser based on block similarity such as block-matching and 3D filtering (BM3D) [13] and weighted nuclear norm minimization (WNNM) [14] enjoy more effective sparsity representation than TV. However, these methods have high computational complexity and often take several hours, while the TV algorithm only takes a few minutes. As a result, BM3D and WNNM are rarely used in Compressed-FLIM, especially when real-time imaging is required. In contrast to conventional denoisers, Deep denoiser networks such as FFDNet [15] and FastDVDnet [16, 17] resolve the common sparsity representation problem in local similarity and motion compensation while enjoying fast computing speed. However, due to limited priors with the training sets, Deep denoiser networks are required to extract artifacts in the reconstruction process, leading to confusing results. To take advantage of both the Deep denoiser network and TV model, H. Qiu et al. proposed a combined denoiser TV+FFDNet and achieved superior performance to previous algorithms [18]. Inspired by combined priors, we further explore a more effective combination of traditional denoisers and Deep denoiser networks. In this paper, we devise a 3DTG V denoiser by exploring the underlying sparsity of signals in space-time and the superiority of the non-convex ℓ(0

We make various simulations based on the CUP framework and determine that the proposed 3DTG V_net prior offers a ~4dB improvement in peak signal-to-noise ratio (PSNR) compared with the TV+FFDNet prior in runner test sets. Meanwhile, the reconstruction artifacts caused by Deep denoiser networks are successfully eliminated. Besides, we conduct a widefield Compressed-FLIM experiment and obtain 70 consecutive high-resolution images within a single snapshot. Compared with the lifetime bias of the reconstructed data with TV+FFDNet, our method provides higher lifetime evaluation accuracy.

2. Principle of compressed-FLIM

A schematic diagram of the compressed ultrafast photography-FLIM (Compressed-FLIM) is illustrated in Fig 1. It comprises three parts: generation of widefield fluorescence signals, data acquisition, and data reconstruction. Unlike the previous scheme [10], we use a transmissive mask rather than the reflective a digital mirror device (DMD) as the spatial encoder.
Fig 1

Schematic diagram of CUP-FLIM.

M1: a pre-designed circular mask with a central cross; M2: the fixed binary mask; BS: beam splitter.

Schematic diagram of CUP-FLIM.

M1: a pre-designed circular mask with a central cross; M2: the fixed binary mask; BS: beam splitter.

2.1 Generation of widefield fluorescence signals

A 515nm femtosecond laser (200fs) beam passes a cylindrical lens into a laser sheet. The laser sheet illuminates a Rhodamine water solution. Behind the Rhodamine solution, a 515nm filter is positioned to filter excitation light. The fluorescence signals pass through a pre-designed circular mask (M1) cut with a cross to highlight spatial recognition, generating shaped fluorescence signals. The diameter of M1 is 35mm.

2.2 Data acquisition

After passing through a lens, the shaped signals are divided into two beams by a beam splitter (BS). One sub-signal is directly detected with an external charge-coupled device (CCD) image sensor (Hamamatsu C11440). The other is spatially encoded through a binary mask M2 and recorded by a Streak Camera (XIOPM 5200). The layout of M2 is a random pattern with a pixel resolution of 250 × 250, and the size of a single-pixel is 20 × 20μm. To ensure entire imaging of the targets, the slit of the Streak Camera is fully open (~5 mm), and the image plane of the Streak Camera is adjusted at M2. In the acquisition section of Streak Camera, the encoded signal undergoes photoelectric conversion at the cathode, then the electric signals at different times are deflected by the slope voltage to various positions on the fluorescent screen. Finally, photons are emitted and collected by the internal CCD (512 × 512 binned pixels; 4 × 4 binning). The size of the binned pixels is 26 × 26 μm. A DMD is the typical encoder in the CUP system. However, for weak fluorescence acquisition, the fixed binary mask [19] significantly improves the signal-to-noise ratio (SNR) by its transmission characteristics. We randomly generated multiple groups of coding layouts through MATLAB, and selected the best coding layout through simulation results. The signals transmittance rate is ultimately set to 25%.

2.3 Data reconstruction

The fluorescence signals can be regarded as a data cube I(x, y, t). In the external CCD view, the cube is directly integrated along the time direction, and the measured data from the CCD can be expressed as E = ∫I(x, y, t)dt. From the perspective of Streak Camera, operator T carries out spatial coding of the cube, and operator S executes the shearing of signals from the coding cube to the tilted coding cube. Ultimately, the accumulation of tilted coding cubes along the time direction is represented by operator C. The entire data acquisition process in Streak Camera view can be described as E = TSCI(x, y, t). Data reconstruction is an ill-condition inverse process. Adding sparsity constraints to the least-squares method realizes the stable reconstruction of the algorithm. The optimization problem of CUP-FLIM can be expressed as: where the first and the second terms are fidelity terms with data collected by the Streak Camera and external CCD, respectively. The last term φ(I) represents the prior used to impose sparsity features to signals while μ and λ are weight parameters. In the next section, we will describe the implementation process of the proposed algorithm and the innovative sophisticated prior.

3. Reconstruction algorithm

3.1 3DTGV priors

Prior plays a key role in the reconstruction algorithms of compressed sensing. The ℓ0 norm prior is the sparsest representation, as it counts the number of nonzero entries in signals. However, it is extremely challenging to process numerically. For solving the dilemma of algorithms without convergence, Donoho. et al. verified the approximate equivalence of the ℓ1 and ℓ0 norms [20]. Formally, the ℓ1 norm minimization can be expressed as In the research area of images, by considering the spatial smoothing properties of natural signals, the generalized form of ℓ1 norm total variation (TV) has been proved to be far sparser when applying the minimization principle of image gradient shown in Fig 2A. The TV prior obeys
Fig 2

Associated elements among the different priors (a)TV (b) TGV (c) 3DTV.

Associated elements among the different priors (a)TV (b) TGV (c) 3DTV. Next, we will briefly introduce three generalized forms (3DTV, TGV, TV) based on the TV prior that strengthen sparsity representation. We define

3.1.1 Stretch in the spatial domain–TGV

Total generalized variation (TGV) is a second-order gradient minimization proposed by Kunisch. et al. [21]. It incorporates more adjacent elements than TV and regards the second-order gradient of images as the sparse coefficients, as Fig 2B illustrates. For mathematical imaging problems, TGV is an effective approach that enhances the details of high-frequency signals and eliminates staircasing effects [22]. It can be expressed as

3.1.2 Stretch in the time domain– 3DTV

The TV prior merely considers image similarity in continuous 2D space but ignores the similarity of adjacent elements [23] in the time direction. 3DTV introduces the 3D sparsity constraint of fluorescence signals shown in Fig 2C, and can be represented as Furthermore, given that the time-domain correlation decreases with the increase of motion scale, we add a time-domain weight parameter τ (0 ≤ τ ≤ 1) to flexibly balance the relevancy between motion scale and time-domain correlation. Therefore, Eq (6) can be rewritten as

3.1.3 ℓ(0

The (0

24]. With superior sparsity performance, the TV prior eliminates artifacts and achieves superior reconstruction results [25, 26]. The TV prior is expressed as By combining the different merits of the three TV-based priors, we proposed the following 3DTGV prior:

3.2 PnP- 3DTGV_net algorithms

In this section, we propose a novel algorithm based on the PnP-framework 3DTG V_net prior by combining 3DTG V prior and Deep denoiser network. We will introduce the overall algorithm flow presented in Fig 3 and Algorithm 1. The sparse signals of interest are rebuilt by determining the minimum solution from the following constrained formula where denotes data measured by Streak Camera. signifies data measured by the external CCD, A and A represent the corresponding projection matrices, respectively, v is an auxiliary variable, and λ is an added weight.
Fig 3

Workflow of the PnP- 3DTGV_net algorithm.

According to the generalized alternating projection (GAP) algorithm [12], we update the fidelity and prior term separately. Fig 3 displays the workflow of the PnP-3DTG V_net algorithm. For each iteration stage, we first apply the Euclidean projection for updating : The update of is a denoising problem, we execute the denoising process by using 3DTGV and FastDVDnet [16], respectively For the 3DTGV update section where For the Deep denoiser network update section Input , , , Given p ∈ (0,1): Initialize for iteration in range(0, 250): Update streak camera’s reconstruction data by ; Update CCD’s reconstruction data by ; Update ; by where ; Deep network denoising: Update ; Update Obtain reconstruction result:

4 Simulation results

In the simulation, we compare the reconstruction performances of six priors (TV, 3DTGV, BM3D, TV+FFDNet, TV+FastDVDnet, and 3DTG V_net) by applying widely-used drop and runner datasets. Each dataset comprises 30 video clips. For initialization, we set ( = ATb, z(0) = 0, λ = 0.07, μ = 0.1, and τ = 0.2. Each algorithm performs 250 iterations independently. For related Deep network, we directly use the FFDNet model and parameters from https://github.com/cszn/KAIR. Besides, the FastDVDnet model and parameters are from https://github.com/m-tassano/fastdvdnet. The drop and runner datasets are from https://github.com/zsm1211/PnP-SCI/tree/master/dataset/simdata/benchmark. Figs 4 and 5 present the reconstructed frames (seen in Visualization 1 and Visualization 2) restored with different priors using the two above-mentioned datasets. The 3DTGV prior achieves superior detailed than the TV prior. Although BM3D has a more rigorous denoising ability than the TV-based methods, excessive smoothing leads to the loss of image detail. The combined priors based on TV and Deep denoiser networks (TV+FFDNet and TV+FastDVDnet) provide better reconstruction contrast and detail than traditional priors but they expose unsatisfactory artifacts in the reconstructed images. Our proposed combined prior 3DTG V_net succeed in eliminating artifacts, leading to more accurate representations of the original images.
Fig 4

The 5th, 15th and 25th reconstructed frames based on simulated datasets: Drop.

Fig 5

The 5th, 15th and 25th reconstructed frames based on simulated datasets: Runner.

We evaluate the quality of the reconstructed images by two indicators: peak signal-to-noise ratio (PSNR) and structural similarity (SSIM). The PSNR can be calculated by where x and y represents the original image and the reconstructed image, respectively. m and n indicates the height and width of the image, respectively. The SSIM can be calculated by where μ and μ represents the mean value of the original image and the reconstructed image, respectively. and are the corresponding variance. σ means the covariance. Table 1 presents the average PSNR and SSIM results. We can conclude that the 3DTG V_net prior outperforms the other priors in both PSNR and SSIM. Significantly, the proposed prior improve the reconstructed accuracy by approximately 4dB (PSNR) over the state-of-the-art TV+FFDNet in runner test sets.
Table 1

Average PSNR and SSIM results.

PriorsDropRunner
PSNRSSIMPSNRSSIM
TV26.900.90124.860.862
3DTGpV27.130.90525.060.874
BM3D26.570.89124.810.862
TV+FFDNet29.200.93426.910.899
TV+FastDVDnet29.760.94227.850.914
3DTGp V_net 31.75 0.958 30.87 0.948

5. Experiments

In the experiments, we record widefield fluorescence data of Rhodamine 6G and Rhodamine B by CUP. The results are reconstructed using both the PnP—TV+FFDNet algorithm and the proposed PnP - 3DTG V_net algorithm. We set the CUP time resolution to be 330 ps. The reconstruction process is implemented in Ubuntu 20.04 with an NVIDIA GeForce GTX 1650Ti GPU. Fig 6A presents the streak camera measurement data for Rhodamine 6G. The scanning direction of the data is from top to bottom. Fig 6B and 6C show the widefield fluorescence data rebuilt by the PnP-TV+FFDNet and PnP - 3DTG V_net algorithms, respectively. Also, the reconstructed movies are shown in Visualization 3 and Visualization 4. By comparing the two sets of data, it is apparent that our proposed algorithm achieves smoother reconstruction results with fewer artifacts.
Fig 6

Measurement and reconstruction data of Rhodamine 6G: (a) Streak Camera image; (b) Reconstructed frames using PnP-TV+FFDNet algorithm; (c) Reconstructed frames using PnP-3DTG V_net algorithm.

Measurement and reconstruction data of Rhodamine 6G: (a) Streak Camera image; (b) Reconstructed frames using PnP-TV+FFDNet algorithm; (c) Reconstructed frames using PnP-3DTG V_net algorithm. In Fig 7A, the measured data of Rhodamine B is displayed. It has a shorter glow duration than Rhodamine 6G. The corresponding rebuild results are shown in Fig 7B and 7C, and the movies are presented in Visualization 5 and Visualization 6. The results indicate that the proposed algorithm achieves better detail reconstruction in various fluorescence environments.
Fig 7

Measurement and reconstruction data of Rhodamine B: (a) Streak Camera image; (b) Reconstructed frames using PnP-TV+FFDNet algorithm; (c) Reconstructed frames using PnP-3DTG V_net algorithm.

Measurement and reconstruction data of Rhodamine B: (a) Streak Camera image; (b) Reconstructed frames using PnP-TV+FFDNet algorithm; (c) Reconstructed frames using PnP-3DTG V_net algorithm. To further analyze the measurement accuracy of widefield fluorescence lifetime, we implement the exponential fitting with the least square method, based on a mono-exponential decay model for each pixel [27]. The measured decay h(t) can be expressed as where A represents the amplitude, τ denotes the lifetime, ε signifies noise, and irf (t) is the instrument response function (IRF) of the measurement system. Since the full width at half maximum (FWMH) of the laser pulse is 200fs, irf (t) can be regarded as a delta function for fluorescence decays with nanosecond lifetimes. Fig 8A and 8B display the two groups of 2D lifetime images rebuilt using the PnP-TV+FFDNet and PnP-3DTG V_net algorithms, respectively. Besides, Fig 9 shows the reconstruction lifetime bias. For Rhodamine 6G (R6G), the mean lifetime and standard deviation of the proposed algorithm are 3.91 ns and 0.57 ns, respectively. Also, the corresponding values for PnP-TV+FFDNet are 4.41 ns and 1.1 ns. For Rhodamine B (RB), the mean lifetime and standard deviation of the proposed algorithm are 1.68 ns and 0.52 ns, while for PnP-TV+FFDNet they are 1.72 ns and 0.54 ns.
Fig 8

Comparison with 2D lifetime images.

Fig 9

Comparison with reconstruction lifetime bias.

In the slit-scanning mode of the Streak Camera, we re-obtain non-superimposed fluorescence lifetime data as a reference. The single exponential fitting results of Rhodamine 6G and Rhodamine B are 3.62 ns and 1.51 ns, respectively. These improved results demonstrate that our proposed PnP-3DTG V_net algorithm produces a bias that is 0.29 ns and 0.17 ns lower than the PnP-TV+FFDNet algorithm.

6. Conclusion

In this study, we propose 3DTGp V_net, a highly effective Compressed-FLIM combined prior. Results from numerous simulations and experiments confirm that our proposed method has better reconstruction performance than the existing algorithms and presents higher evaluation accuracy for wide-field FLIM. Besides, this study further confirms that combined priors can effectively complement the advantages of traditional priors and Deep denoiser networks to improve the reconstruction performance of compressive video imaging technology. Lastly, it is noted that our algorithm is a general framework and demanding to relevant SCI systems.

Drop.

(AVI) Click here for additional data file.

Runner.

(AVI) Click here for additional data file.

R6G(3DTGpV_net).

(AVI) Click here for additional data file.

R6G(TV+FFDnet).

(AVI) Click here for additional data file.

RB(3DTGpV_net).

(AVI) Click here for additional data file.

RB(TV+FFDnet).

(AVI) Click here for additional data file. 1 Apr 2022
PONE-D-22-04257
Compressed fluorescence lifetime imaging via combined TV-based and Deep Priors
PLOS ONE Dear Dr. jinshou, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.
 
Please submit your revised manuscript by May 16 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript:
A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Li Zeng Academic Editor PLOS ONE Journal Requirements: When submitting your revision, we need you to address these additional requirements. 1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf 2. In the Methods section of your manuscript, please provide full information on the location of the 'drop' and 'runner' datasets. This can take the form of a literature reference, or a URL link to the location of each dataset. 3. We note that the grant information you provided in the ‘Funding Information’ and ‘Financial Disclosure’ sections do not match. When you resubmit, please ensure that you provide the correct grant numbers for the awards you received for your study in the ‘Funding Information’ section. 4. Thank you for stating the following in the Funding Section of your manuscript: "This work was supported by the Scientific Instrument Developing Project of the Chinese Academy of Sciences (Grant No. GJJSTD20220006), the Youth Program of the National Natural Science Foundation of China (Grant No. 12105360, 62075236), the Science Foundation of the Chinese Academy of Sciences (Grant No. CXJJ-21S006), the Youth Innovation Promotion Association CAS (Grant No. 2020397), and the Rising research star of Shaanxi Province (Grant No. 2021SR5061)." We note that you have provided funding information that is not currently declared in your Funding Statement. However, funding information should not appear in the Funding section or other areas of your manuscript. We will only publish funding information present in the Funding Statement section of the online submission form. Please remove any funding-related text from the manuscript and let us know how you would like to update your Funding Statement. Currently, your Funding Statement reads as follows: "Initials of the authors who received each award" Please include your amended statements within your cover letter; we will change the online submission form on your behalf. 5. In your Data Availability statement, you have not specified where the minimal data set underlying the results described in your manuscript can be found. PLOS defines a study's minimal data set as the underlying data used to reach the conclusions drawn in the manuscript and any additional data required to replicate the reported study findings in their entirety. All PLOS journals require that the minimal data set be made fully available. For more information about our data policy, please see http://journals.plos.org/plosone/s/data-availability. Upon re-submitting your revised manuscript, please upload your study’s minimal underlying data set as either Supporting Information files or to a stable, public repository and include the relevant URLs, DOIs, or accession numbers within your revised cover letter. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories. Any potentially identifying patient information must be fully anonymized. Important: If there are ethical or legal restrictions to sharing your data publicly, please explain these restrictions in detail. Please see our guidelines for more information on what we consider unacceptable restrictions to publicly sharing data: http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions. Note that it is not acceptable for the authors to be the sole named individuals responsible for ensuring data access. We will update your Data Availability statement to reflect the information you provide in your cover letter. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Partly ********** 2. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes ********** 3. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: No ********** 4. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: No ********** 5. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: This paper by Choa Ji et al describes a compressed sensing technique for fluorescence lifetime imaging. They propose a novel reconstruction technique that uses a novel prior and this is compared to state of the art techniques. Unfortunately the quality of the writing and some of the methods contained within the paper are not of sufficient standard to fully judge its merits. I therefore recommend a major revision that addresses the points below. The mathematics is quite sparse, consisting mainly of variations of the prior. I suggest that the authors include text on how this new work fits into the "Plug and play" framework. For publication in PLOS ONE (a journal with a non-specialist readership) I would expect more background information. Please consider if it is possible to recreate the results from the information provided. Provide code in an external repository where possible. The ground truth for the 2 simulated examples is not shown. The example experiment contains only Rhodamine 6G are therefore consists of a single lifetime value. The authors should demonstrate that multiple lifetimes within the same scene can be determined. Please add an additional experiment. There are numerous mistakes in the English writing (too many to list) that make it very hard to understand. This has to be improved before a more thorough review can be performed. ********** 6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step. 20 May 2022 We would like to express our sincere thanks to the Editor and Reviewer for the constructive and positive comments. Replies to Editor Comment 1: The mathematics is quite sparse, consisting mainly of variations of the prior. I suggest that the authors include text on how this new work fits into the "Plug and play" framework. Answer: We added a text “Algorithm 1. PnP-〖3DTG〗_p V¬_net framework” behind the fig 3 to demonstrate the fitting workflow between the priors and "Plug and play" framework. Comment 2: For publication in PLOS ONE (a journal with a non-specialist readership) I would expect more background information. Please consider if it is possible to recreate the results from the information provided. Provide code in an external repository where possible. Answer: We added more detailed research background at the beginning of the manuscript introduction. In addition, we have provided all visualization results involving simulations and experiments (seen in Visualization 1-6). As it involves subsequent research, the code will be disclosed later. Comment 3: The ground truth for the 2 simulated examples is not shown. The example experiment contains only Rhodamine 6G are therefore consists of a single lifetime value. The authors should demonstrate that multiple lifetimes within the same scene can be determined. Please add an additional experiment. Answer: We conducted another Compressed-FLIM experiment by using Rhodamine B. Moreover, we confirmed that our algorithm can obtain better reconstruction effect and more accurate lifetime evaluation under different fluorescent samples. Comment 4: There are numerous mistakes in the English writing (too many to list) that make it very hard to understand. This has to be improved before a more thorough review can be performed. Answer: Thanks for your suggestion. We have tried our best to polish the language in the revised manuscript. These changes will not influence the content and framework of the paper. And here we did not list the changes but marked in revisions mode in revised paper. Submitted filename: rebuttal letter.docx Click here for additional data file. 17 Jun 2022
PONE-D-22-04257R1
Compressed fluorescence lifetime imaging via combined TV-based and Deep Priors
PLOS ONE Dear Dr. jinshou, Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process. Please submit your revised manuscript by Aug 01 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file. Please include the following items when submitting your revised manuscript:
If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter. A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'. A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'. An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'. If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols. We look forward to receiving your revised manuscript. Kind regards, Li Zeng Academic Editor PLOS ONE Journal Requirements: Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Any changes to the reference list should be mentioned in the rebuttal letter that accompanies your revised manuscript. If you need to cite a retracted article, indicate the article’s retracted status in the References list and also include a citation and full reference for the retraction notice. [Note: HTML markup is below. Please do not edit.] Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: (No Response) ********** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes ********** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: I Don't Know ********** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: No ********** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes ********** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: This manuscript by Choa Ji et al describes a compressed sensing technique for fluorescence lifetime imaging. This is the second time I have reviewed this paper after major revision. This manuscript is better written and much more understandable. I appreciate the extra background information and detail provided to put the proposed algorithm in context. There has also been significant revision of the methods section and the addition of new experimental results. The authors chose to include a new experiment with a different substance. It would have been better to use a sample that has at least two different regions, containing different substances with different lifetimes, as I suggested in the first review. There are minor additions still required before I can recommend publication. The ground truth for the simulated images are still not included. Figures 4 and 5 show the results of 6 algorithms, alongside those results I need to see the ground truth images from the original dataset. The test data sets are "runner" and "drop". Please describe how these were acquired. What hardware was used etc. Abstract: By "high frames", do you mean "high frame rates"? Abstract: What does "large-scale" mean? Please clarify in the manuscript. Abstract: It is not clear what "runner test sets" means, please clarify or replace with "test data sets". Figure 1: Please describe the acronyms (M1 etc) in the figure legend. Line 95: The purpose of M1 is not clear. Is it crucial to the algorithm or just to provide some arbitrary spatial structure? Line 110: Remove "In the CUP system" and start the sentence with "A DMD is..." Line 112: Please clarify the sentence, "The binary coding is isolated..." Line 141: Please clarify what is meant by "continuous spatial feathers". Line 148: "We can Assume" should be "We define" Line 237: Please define PSNR and SSIM. ********** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No ********** [NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.] While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.
24 Jun 2022 Replies to Journal Comment 1: Please review your reference list to ensure that it is complete and correct. If you have cited papers that have been retracted, please include the rationale for doing so in the manuscript text, or remove these references and replace them with relevant current references. Answer: We have checked the reference list and confirmed that it is complete and correct, and there is no retracted article. Replies to Reviewer #1 Comment 1: The ground truth for the simulated images are still not included. Figures 4 and 5 show the results of 6 algorithms, alongside those results I need to see the ground truth images from the original dataset. The test data sets are "runner" and "drop". Please describe how these were acquired. What hardware was used etc. Answer: We added the original images labeled ‘Ground Truth’ in Figures 4 and 5, and we added the test data sets acquisition links in 219 lines of the manuscript. Moreover, in 262 lines of the manuscript, we describe the system environment for algorithm execution Comment 2: Abstract: By "high frames", do you mean "high frame rates"? Answer: The "high frames" is intended to express the number of imaging frames. But the "high frames" is an inappropriate description. We have changed it to the “deep frame sequences”. Comment 3: Abstract: What does "large-scale" mean? Please clarify in the manuscript. Answer: For the problem of image compression and reconstruction, the “large-scale” generally refers to the deep compressed frame sequences. For example, the number of compressed images exceeds 30. The term “large-scale” can be found in reference “Plug-and-Play Algorithms for Large-Scale Snapshot Compressive Imaging”. For a more reasonable description, we change the “large-scale FLIM” to the “large-scale FLIM problem” in the manuscript. Comment 4: Abstract: It is not clear what "runner test sets" means, please clarify or replace with "test data sets". Answer: We changed the "runner test sets" with the "test data sets". Comment 5: Figure 1: Please describe the acronyms (M1 etc) in the figure legend. Line 95: The purpose of M1 is not clear. Is it crucial to the algorithm or just to provide some arbitrary spatial structure? Answer: We added the acronyms description in the figure1 legend. M1 is only to highlight the 2D spatial distribution and has nothing to do with the algorithms. Comment 6: Line 110: Remove "In the CUP system" and start the sentence with "A DMD is..." Answer: We removed the "In the CUP system" and modified our grammar mistakes. Comment 7: Line 112: Please clarify the sentence, "The binary coding is isolated..." Answer: We changed the original sentence to " We randomly generated multiple groups of coding layouts through MATLAB, and selected the best coding layout by simulation results " Comment 8: Line 141: Please clarify what is meant by "continuous spatial feathers". Answer: The "continuous spatial feathers" is intended to express the smoothing properties of natural signals. We replaced the "continuous spatial feathers" with the “spatial smoothing properties” Comment 9: Line 148: "We can Assume" should be "We define" Answer: We have replaced the "We can Assume" with the "We define". Comment 10: Line 237: Please define PSNR and SSIM. Answer: We added the formulas and descriptions of the PSNR and SSIM at line 238 of the manuscript Submitted filename: Response to Reviewers.docx Click here for additional data file. 1 Jul 2022 Compressed fluorescence lifetime imaging via combined TV-based and Deep Priors PONE-D-22-04257R2 Dear Dr. jinshou, We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements. Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication. An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org. If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org. Kind regards, Li Zeng Academic Editor PLOS ONE Additional Editor Comments (optional): Reviewers' comments: Reviewer's Responses to Questions Comments to the Author 1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation. Reviewer #1: All comments have been addressed ********** 2. Is the manuscript technically sound, and do the data support the conclusions? The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented. Reviewer #1: Yes ********** 3. Has the statistical analysis been performed appropriately and rigorously? Reviewer #1: Yes ********** 4. Have the authors made all data underlying the findings in their manuscript fully available? The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified. Reviewer #1: Yes ********** 5. Is the manuscript presented in an intelligible fashion and written in standard English? PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here. Reviewer #1: Yes ********** 6. Review Comments to the Author Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters) Reviewer #1: Thank you for your revisions. I now recommend publication. Please consider making your code available according to the policy: https://journals.plos.org/plosone/s/materials-software-and-code-sharing ********** 7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files. If you choose “no”, your identity will remain anonymous but your review may still be made public. Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy. Reviewer #1: No ********** 3 Aug 2022 PONE-D-22-04257R2 Compressed fluorescence lifetime imaging via combined TV-based and Deep Priors Dear Dr. Tian: I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department. If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org. If we can help with anything else, please email us at plosone@plos.org. Thank you for submitting your work to PLOS ONE and supporting open access. Kind regards, PLOS ONE Editorial Office Staff on behalf of Professor Li Zeng Academic Editor PLOS ONE
  13 in total

1.  FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising.

Authors:  Kai Zhang; Wangmeng Zuo; Lei Zhang
Journal:  IEEE Trans Image Process       Date:  2018-05-25       Impact factor: 10.856

2.  A 512×512 SPAD Image Sensor with Integrated Gating for Widefield FLIM.

Authors:  Arin C Ulku; Claudio Bruschini; Ivan Michel Antolovic; Edoardo Charbon; Yung Kuo; Rinat Ankri; Shimon Weiss; Xavier Michalet
Journal:  IEEE J Sel Top Quantum Electron       Date:  2018-08-28       Impact factor: 4.544

3.  Dead-time correction of fluorescence lifetime measurements and fluorescence lifetime imaging.

Authors:  Sebastian Isbaner; Narain Karedla; Daja Ruhlandt; Simon Christoph Stein; Anna Chizhik; Ingo Gregor; Jörg Enderlein
Journal:  Opt Express       Date:  2016-05-02       Impact factor: 3.894

4.  Hyperspectrally Compressed Ultrafast Photography.

Authors:  Chengshuai Yang; Fengyan Cao; Dalong Qi; Yilin He; Pengpeng Ding; Jiali Yao; Tianqing Jia; Zhenrong Sun; Shian Zhang
Journal:  Phys Rev Lett       Date:  2020-01-17       Impact factor: 9.161

5.  Ultra-high-speed PLIF imaging for simultaneous visualization of multiple species in turbulent flames.

Authors:  Zhenkan Wang; Panagiota Stamatoglou; Zheming Li; Marcus Aldén; Mattias Richter
Journal:  Opt Express       Date:  2017-11-27       Impact factor: 3.894

6.  High-speed compressed-sensing fluorescence lifetime imaging microscopy of live cells.

Authors:  Yayao Ma; Youngjae Lee; Catherine Best-Popescu; Liang Gao
Journal:  Proc Natl Acad Sci U S A       Date:  2021-01-19       Impact factor: 12.779

7.  Constrained Total Generalized p-Variation Minimization for Few-View X-Ray Computed Tomography Image Reconstruction.

Authors:  Hanming Zhang; Linyuan Wang; Bin Yan; Lei Li; Ailong Cai; Guoen Hu
Journal:  PLoS One       Date:  2016-02-22       Impact factor: 3.240

8.  A new development of non-local image denoising using fixed-point iteration for non-convex ℓp sparse optimization.

Authors:  Shuting Cai; Kun Liu; Ming Yang; Jianliang Tang; Xiaoming Xiong; Mingqing Xiao
Journal:  PLoS One       Date:  2018-12-12       Impact factor: 3.240

9.  Fluorescence lifetime imaging with a megapixel SPAD camera and neural network lifetime estimation.

Authors:  Vytautas Zickus; Ming-Lo Wu; Kazuhiro Morimoto; Valentin Kapitany; Areeba Fatima; Alex Turpin; Robert Insall; Jamie Whitelaw; Laura Machesky; Claudio Bruschini; Daniele Faccio; Edoardo Charbon
Journal:  Sci Rep       Date:  2020-12-02       Impact factor: 4.379

Review 10.  FLIM as a Promising Tool for Cancer Diagnosis and Treatment Monitoring.

Authors:  Yuzhen Ouyang; Yanping Liu; Zhiming M Wang; Zongwen Liu; Minghua Wu
Journal:  Nanomicro Lett       Date:  2021-06-03
View more

北京卡尤迪生物科技股份有限公司 © 2022-2023.