Cheng Yuan1, Qing Shi2, Xinyun Huang3, Li Wang2, Yang He2, Biao Li4, Weili Zhao5, Dahong Qian6. 1. School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, 200040, China. 2. Shanghai Institute of Hematology, State Key Laboratory of Medical Genomics, National Research Center for Translational Medicine at Shanghai, Ruijin Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, 200025, China. 3. Department of Nuclear Medicine, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200025, China. 4. Department of Nuclear Medicine, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, 200025, China. lb10363@rjh.com.cn. 5. Shanghai Institute of Hematology, State Key Laboratory of Medical Genomics, National Research Center for Translational Medicine at Shanghai, Ruijin Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, 200025, China. zhao.weili@yahoo.com. 6. School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, 200040, China. dahong.qian@sjtu.edu.cn.
Abstract
OBJECTIVES: The prediction of primary treatment failure (PTF) is necessary for patients with diffuse large B-cell lymphoma (DLBCL) since it serves as a prominent means for improving front-line outcomes. Using interim 18F-fluoro-2-deoxyglucose ([18F]FDG) positron emission tomography/computed tomography (PET/CT) imaging data, we aimed to construct multimodal deep learning (MDL) models to predict possible PTF in low-risk DLBCL. METHODS: Initially, 205 DLBCL patients undergoing interim [18F]FDG PET/CT scans and the front-line standard of care were included in the primary dataset for model development. Then, 44 other patients were included in the external dataset for generalization evaluation. Based on the powerful backbone of the Conv-LSTM network, we incorporated five different multimodal fusion strategies (pixel intermixing, separate channel, separate branch, quantitative weighting, and hybrid learning) to make full use of PET/CT features and built five corresponding MDL models. Moreover, we found the best model, that is, the hybrid learning model, and optimized it by integrating the contrastive training objective to further improve its prediction performance. RESULTS: The final model with contrastive objective optimization, named the contrastive hybrid learning model, performed best, with an accuracy of 91.22% and an area under the receiver operating characteristic curve (AUC) of 0.926, in the primary dataset. In the external dataset, its accuracy and AUC remained at 88.64% and 0.925, respectively, indicating its good generalization ability. CONCLUSIONS: The proposed model achieved good performance, validated the predictive value of interim PET/CT, and holds promise for directing individualized clinical treatment. KEY POINTS: • The proposed multimodal models achieved accurate prediction of primary treatment failure in DLBCL patients. • Using an appropriate feature-level fusion strategy can make the same class close to each other regardless of the modal heterogeneity of the data source domain and positively impact the prediction performance. • Deep learning validated the predictive value of interim PET/CT in a way that exceeded human capabilities.
OBJECTIVES: The prediction of primary treatment failure (PTF) is necessary for patients with diffuse large B-cell lymphoma (DLBCL) since it serves as a prominent means for improving front-line outcomes. Using interim 18F-fluoro-2-deoxyglucose ([18F]FDG) positron emission tomography/computed tomography (PET/CT) imaging data, we aimed to construct multimodal deep learning (MDL) models to predict possible PTF in low-risk DLBCL. METHODS: Initially, 205 DLBCL patients undergoing interim [18F]FDG PET/CT scans and the front-line standard of care were included in the primary dataset for model development. Then, 44 other patients were included in the external dataset for generalization evaluation. Based on the powerful backbone of the Conv-LSTM network, we incorporated five different multimodal fusion strategies (pixel intermixing, separate channel, separate branch, quantitative weighting, and hybrid learning) to make full use of PET/CT features and built five corresponding MDL models. Moreover, we found the best model, that is, the hybrid learning model, and optimized it by integrating the contrastive training objective to further improve its prediction performance. RESULTS: The final model with contrastive objective optimization, named the contrastive hybrid learning model, performed best, with an accuracy of 91.22% and an area under the receiver operating characteristic curve (AUC) of 0.926, in the primary dataset. In the external dataset, its accuracy and AUC remained at 88.64% and 0.925, respectively, indicating its good generalization ability. CONCLUSIONS: The proposed model achieved good performance, validated the predictive value of interim PET/CT, and holds promise for directing individualized clinical treatment. KEY POINTS: • The proposed multimodal models achieved accurate prediction of primary treatment failure in DLBCL patients. • Using an appropriate feature-level fusion strategy can make the same class close to each other regardless of the modal heterogeneity of the data source domain and positively impact the prediction performance. • Deep learning validated the predictive value of interim PET/CT in a way that exceeded human capabilities.
Authors: Michael Crump; Sattva S Neelapu; Umar Farooq; Eric Van Den Neste; John Kuruvilla; Jason Westin; Brian K Link; Annette Hay; James R Cerhan; Liting Zhu; Sami Boussetta; Lei Feng; Matthew J Maurer; Lynn Navale; Jeff Wiezorek; William Y Go; Christian Gisselbrecht Journal: Blood Date: 2017-08-03 Impact factor: 22.113
Authors: Frederick L Locke; Armin Ghobadi; Caron A Jacobson; David B Miklos; Lazaros J Lekakis; Olalekan O Oluwole; Yi Lin; Ira Braunschweig; Brian T Hill; John M Timmerman; Abhinav Deol; Patrick M Reagan; Patrick Stiff; Ian W Flinn; Umar Farooq; Andre Goy; Peter A McSweeney; Javier Munoz; Tanya Siddiqi; Julio C Chavez; Alex F Herrera; Nancy L Bartlett; Jeffrey S Wiezorek; Lynn Navale; Allen Xue; Yizhou Jiang; Adrian Bot; John M Rossi; Jenny J Kim; William Y Go; Sattva S Neelapu Journal: Lancet Oncol Date: 2018-12-02 Impact factor: 41.316
Authors: Malik E Juweid; Sigrid Stroobants; Otto S Hoekstra; Felix M Mottaghy; Markus Dietlein; Ali Guermazi; Gregory A Wiseman; Lale Kostakoglu; Klemens Scheidhauer; Andreas Buck; Ralph Naumann; Karoline Spaepen; Rodney J Hicks; Wolfgang A Weber; Sven N Reske; Markus Schwaiger; Lawrence H Schwartz; Josee M Zijlstra; Barry A Siegel; Bruce D Cheson Journal: J Clin Oncol Date: 2007-01-22 Impact factor: 44.544
Authors: P Feugier; A Van Hoof; C Sebban; P Solal-Celigny; R Bouabdallah; C Fermé; B Christian; E Lepage; H Tilly; F Morschhauser; P Gaulard; G Salles; A Bosly; C Gisselbrecht; F Reyes; B Coiffier Journal: J Clin Oncol Date: 2005-05-02 Impact factor: 44.544
Authors: Nagesh Kalakonda; Marie Maerevoet; Federica Cavallo; George Follows; Andre Goy; Joost S P Vermaat; Olivier Casasnovas; Nada Hamad; Josée M Zijlstra; Sameer Bakhshi; Reda Bouabdallah; Sylvain Choquet; Ronit Gurion; Brian Hill; Ulrich Jaeger; Juan Manuel Sancho; Michael Schuster; Catherine Thieblemont; Fátima De la Cruz; Miklos Egyed; Sourav Mishra; Fritz Offner; Theodoros P Vassilakopoulos; Krzysztof Warzocha; Daniel McCarthy; Xiwen Ma; Kelly Corona; Jean-Richard Saint-Martin; Hua Chang; Yosef Landesman; Anita Joshi; Hongwei Wang; Jatin Shah; Sharon Shacham; Michael Kauffman; Eric Van Den Neste; Miguel A Canales Journal: Lancet Haematol Date: 2020-07 Impact factor: 18.959
Authors: Zijun Y Xu-Monette; Lin Wu; Carlo Visco; Yu Chuan Tai; Alexander Tzankov; Wei-min Liu; Santiago Montes-Moreno; Karen Dybkaer; April Chiu; Attilio Orazi; Youli Zu; Govind Bhagat; Kristy L Richards; Eric D Hsi; X Frank Zhao; William W L Choi; Xiaoying Zhao; J Han van Krieken; Qin Huang; Jooryung Huh; Weiyun Ai; Maurilio Ponzoni; Andrés J M Ferreri; Fan Zhou; Brad S Kahl; Jane N Winter; Wei Xu; Jianyong Li; Ronald S Go; Yong Li; Miguel A Piris; Michael B Møller; Roberto N Miranda; Lynne V Abruzzo; L Jeffrey Medeiros; Ken H Young Journal: Blood Date: 2012-09-05 Impact factor: 22.113