Yi Sun1,2, Sixian You1,3, Xiaoxi Du1,2, Allison Spaulding1, Z George Liu4, Eric J Chaney1, Darold R Spillman1, Marina Marjanovic1,3,4, Haohua Tu1, Stephen A Boppart1,2,3,4,5. 1. Beckman Institute for Advanced Science and Technology, University of Illinois at Urbana-Champaign, Urbana, IL, USA. 2. Department of Electrical and Computer Engineering, University of Illinois at Urbana-Champaign, Urbana, IL, USA. 3. Department of Bioengineering, University of Illinois at Urbana-Champaign, Urbana, IL, USA. 4. Carle Illinois College of Medicine, University of Illinois at Urbana-Champaign, Urbana, IL, USA. 5. Cancer Center at Illinois, University of Illinois at Urbana-Champaign, Urbana, IL, USA.
Abstract
BACKGROUND: The current gold-standard formalin-fixed and paraffin-embedded (FFPE) histology typically requires several days for tissue fixing, embedding, sectioning, and staining to provide depth-resolved tissue feature visualization. During these time- and labor- intense processes, the in vivo tissue dynamics and three-dimensional structures undergo inevitable loss and distortion. METHODS: A simultaneous label-free autofluorescence multiharmonic (SLAM) microscope is used to conduct ex vivo and in vivo imaging of fresh human and rat tissues. Four nonlinear optical imaging modalities are integrated into this SLAM microscope, including second harmonic generation (SHG), two-photon fluorescence (2PF), third harmonic generation (THG), and three-photon fluorescence (3PF). By imaging fresh human and rat tissues without any tissue processing or staining, various biological tissue features are effectively visualized by one or multiple imaging modalities of the SLAM microscope. In particular, some of the most essential features in hematoxylin and eosin (H&E)-stained histology, such as collagen fibers and nuclei, are also present in the SLAM microscopy images with good contrast. Because nuclei are evident from negative contrast, the nuclei are segmented from the SLAM images using deep learning. Finally, a color-transforming algorithm is developed to convert the grey-scale images acquired by the SLAM microscope to the virtually H&E-stained histology-like images. The converted histology-like images are later compared with the FFPE histology at the same tissue site. In addition, the nuclear-to-cytoplasmic ratios (N/C ratios) of the cells in the SLAM image are quantified, which has diagnostic relevance for cancer. RESULTS: Various histological correlations are identified with high similarities for the color-converted histology-like SLAM microscopy images. By applying the color transforming algorithm on real-time SLAM image sequences and 3D SLAM image stacks, we report, for the first time and to the best our knowledge, real-time 3D histology-like imaging. Furthermore, the quantified N/C ratio of the cells in the SLAM image are overlaid on the converted histology-like image as a new image contrast. CONCLUSIONS: We demonstrated real-time 3D histology-like imaging and its future potential using SLAM microscopy aided by color remapping and deep-learning-based feature segmentation. 2020 Quantitative Imaging in Medicine and Surgery. All rights reserved.
BACKGROUND: The current gold-standard formalin-fixed and paraffin-embedded (FFPE) histology typically requires several days for tissue fixing, embedding, sectioning, and staining to provide depth-resolved tissue feature visualization. During these time- and labor- intense processes, the in vivo tissue dynamics and three-dimensional structures undergo inevitable loss and distortion. METHODS: A simultaneous label-free autofluorescence multiharmonic (SLAM) microscope is used to conduct ex vivo and in vivo imaging of fresh human and rat tissues. Four nonlinear optical imaging modalities are integrated into this SLAM microscope, including second harmonic generation (SHG), two-photon fluorescence (2PF), third harmonic generation (THG), and three-photon fluorescence (3PF). By imaging fresh human and rat tissues without any tissue processing or staining, various biological tissue features are effectively visualized by one or multiple imaging modalities of the SLAM microscope. In particular, some of the most essential features in hematoxylin and eosin (H&E)-stained histology, such as collagen fibers and nuclei, are also present in the SLAM microscopy images with good contrast. Because nuclei are evident from negative contrast, the nuclei are segmented from the SLAM images using deep learning. Finally, a color-transforming algorithm is developed to convert the grey-scale images acquired by the SLAM microscope to the virtually H&E-stained histology-like images. The converted histology-like images are later compared with the FFPE histology at the same tissue site. In addition, the nuclear-to-cytoplasmic ratios (N/C ratios) of the cells in the SLAM image are quantified, which has diagnostic relevance for cancer. RESULTS: Various histological correlations are identified with high similarities for the color-converted histology-like SLAM microscopy images. By applying the color transforming algorithm on real-time SLAM image sequences and 3D SLAM image stacks, we report, for the first time and to the best our knowledge, real-time 3D histology-like imaging. Furthermore, the quantified N/C ratio of the cells in the SLAM image are overlaid on the converted histology-like image as a new image contrast. CONCLUSIONS: We demonstrated real-time 3D histology-like imaging and its future potential using SLAM microscopy aided by color remapping and deep-learning-based feature segmentation. 2020 Quantitative Imaging in Medicine and Surgery. All rights reserved.
Entities:
Keywords:
Digital pathology; cancer diagnostics; deep learning; digital staining; label-free; virtual histology
Authors: Melissa C Skala; Kristin M Riching; Annette Gendron-Fitzpatrick; Jens Eickhoff; Kevin W Eliceiri; John G White; Nirmala Ramanujam Journal: Proc Natl Acad Sci U S A Date: 2007-11-27 Impact factor: 11.205
Authors: Yair Rivenson; Hongda Wang; Zhensong Wei; Kevin de Haan; Yibo Zhang; Yichen Wu; Harun Günaydın; Jonathan E Zuckerman; Thomas Chong; Anthony E Sisk; Lindsey M Westbrook; W Dean Wallace; Aydogan Ozcan Journal: Nat Biomed Eng Date: 2019-03-04 Impact factor: 25.671
Authors: Sarah J Erickson-Bhatt; Ryan M Nolan; Nathan D Shemonski; Steven G Adie; Jeffrey Putney; Donald Darga; Daniel T McCormick; Andrew J Cittadine; Adam M Zysk; Marina Marjanovic; Eric J Chaney; Guillermo L Monroy; Fredrick A South; Kimberly A Cradock; Z George Liu; Magesh Sundaram; Partha S Ray; Stephen A Boppart Journal: Cancer Res Date: 2015-09-15 Impact factor: 12.701
Authors: Sixian You; Ronit Barkalifa; Eric J Chaney; Haohua Tu; Jaena Park; Janet Elise Sorrells; Yi Sun; Yuan-Zhi Liu; Lin Yang; Danny Z Chen; Marina Marjanovic; Saurabh Sinha; Stephen A Boppart Journal: Proc Natl Acad Sci U S A Date: 2019-11-15 Impact factor: 11.205
Authors: Andrew J Bower; Joanne Li; Eric J Chaney; Marina Marjanovic; Darold R Spillman; Stephen A Boppart Journal: Optica Date: 2018-10-16 Impact factor: 11.104
Authors: Terence T W Wong; Ruiying Zhang; Pengfei Hai; Chi Zhang; Miguel A Pleitez; Rebecca L Aft; Deborah V Novack; Lihong V Wang Journal: Sci Adv Date: 2017-05-17 Impact factor: 14.136
Authors: Benjamin R Ecclestone; Kevan Bell; Sarah Sparkes; Deepak Dinakaran; John R Mackey; Parsin Haji Reza Journal: Sci Rep Date: 2022-05-19 Impact factor: 4.996
Authors: Rishyashring R Iyer; Janet E Sorrells; Lingxiao Yang; Eric J Chaney; Darold R Spillman; Brian E Tibble; Carlos A Renteria; Haohua Tu; Mantas Žurauskas; Marina Marjanovic; Stephen A Boppart Journal: Sci Rep Date: 2022-03-02 Impact factor: 4.379