Sui Peng1, Yihao Liu2, Weiming Lv3, Longzhong Liu4, Qian Zhou1, Hong Yang5, Jie Ren6, Guangjian Liu7, Xiaodong Wang8, Xuehua Zhang9, Qiang Du10, Fangxing Nie10, Gao Huang10, Yuchen Guo11, Jie Li3, Jinyu Liang12, Hangtong Hu12, Han Xiao12, Zelong Liu12, Fenghua Lai13, Qiuyi Zheng13, Haibo Wang1, Yanbing Li13, Erik K Alexander14, Wei Wang15, Haipeng Xiao16. 1. Clinical Trials Unit, The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China. 2. Clinical Trials Unit, The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China; Department of Endocrinology, The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China; Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China. 3. Department of Breast and Thyroid Surgery, The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China. 4. Department of Ultrasound, Sun Yat-sen University Cancer Center, State Key Laboratory of Oncology in South China, Guangzhou, China. 5. Department of Medical Ultrasound, the First Affiliated Hospital of Guangxi Medical University, Nanning, China. 6. Department of Medical Ultrasonics, the Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China. 7. Department of Medical Ultrasonics, the Sixth Affiliated Hospital of Sun Yat-sen University, Guangzhou, China. 8. Department of Medical Ultrasonics, the First Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangzhou, China. 9. Department of Ultrasound, the Guangzhou Army General Hospital, Guangzhou, China. 10. Xiaobaishiji, Beijing, China. 11. Institute for Brain and Cognitive Sciences, Tsinghua University, Beijing, China. 12. Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China. 13. Department of Endocrinology, The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China. 14. Thyroid Section, Brigham & Women's Hospital, Harvard Medical School, Boston, MA, USA. Electronic address: ekalexander@bwh.harvard.edu. 15. Department of Medical Ultrasonics, Institute of Diagnostic and Interventional Ultrasound, The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China. Electronic address: wangw73@mail.sysu.edu.cn. 16. Department of Endocrinology, The First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China. Electronic address: xiaohp@mail.sysu.edu.cn.
Abstract
BACKGROUND: Strategies for integrating artificial intelligence (AI) into thyroid nodule management require additional development and testing. We developed a deep-learning AI model (ThyNet) to differentiate between malignant tumours and benign thyroid nodules and aimed to investigate how ThyNet could help radiologists improve diagnostic performance and avoid unnecessary fine needle aspiration. METHODS: ThyNet was developed and trained on 18 049 images of 8339 patients (training set) from two hospitals (the First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China, and Sun Yat-sen University Cancer Center, Guangzhou, China) and tested on 4305 images of 2775 patients (total test set) from seven hospitals (the First Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangzhou, China; the Sixth Affiliated Hospital of Sun Yat-sen University, Guangzhou, China; the Guangzhou Army General Hospital, Guangzhou, China; the Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China; the First Affiliated Hospital of Sun Yat-sen University; Sun Yat-sen University Cancer Center; and the First Affiliated Hospital of Guangxi Medical University, Nanning, China) in three stages. All nodules in the training and total test set were pathologically confirmed. The diagnostic performance of ThyNet was first compared with 12 radiologists (test set A); a ThyNet-assisted strategy, in which ThyNet assisted diagnoses made by radiologists, was developed to improve diagnostic performance of radiologists using images (test set B); the ThyNet assisted strategy was then tested in a real-world clinical setting (using images and videos; test set C). In a simulated scenario, the number of unnecessary fine needle aspirations avoided by ThyNet-assisted strategy was calculated. FINDINGS: The area under the receiver operating characteristic curve (AUROC) for accurate diagnosis of ThyNet (0·922 [95% CI 0·910-0·934]) was significantly higher than that of the radiologists (0·839 [0·834-0·844]; p<0·0001). Furthermore, ThyNet-assisted strategy improved the pooled AUROC of the radiologists from 0·837 (0·832-0·842) when diagnosing without ThyNet to 0·875 (0·871-0·880; p<0·0001) with ThyNet for reviewing images, and from 0·862 (0·851-0·872) to 0·873 (0·863-0·883; p<0·0001) in the clinical test, which used images and videos. In the simulated scenario, the number of fine needle aspirations decreased from 61·9% to 35·2% using the ThyNet-assisted strategy, while missed malignancy decreased from 18·9% to 17·0%. INTERPRETATION: The ThyNet-assisted strategy can significantly improve the diagnostic performance of radiologists and help reduce unnecessary fine needle aspirations for thyroid nodules. FUNDING: National Natural Science Foundation of China and Guangzhou Science and Technology Project.
BACKGROUND: Strategies for integrating artificial intelligence (AI) into thyroid nodule management require additional development and testing. We developed a deep-learning AI model (ThyNet) to differentiate between malignant tumours and benign thyroid nodules and aimed to investigate how ThyNet could help radiologists improve diagnostic performance and avoid unnecessary fine needle aspiration. METHODS: ThyNet was developed and trained on 18 049 images of 8339 patients (training set) from two hospitals (the First Affiliated Hospital of Sun Yat-sen University, Guangzhou, China, and Sun Yat-sen University Cancer Center, Guangzhou, China) and tested on 4305 images of 2775 patients (total test set) from seven hospitals (the First Affiliated Hospital of Guangzhou University of Chinese Medicine, Guangzhou, China; the Sixth Affiliated Hospital of Sun Yat-sen University, Guangzhou, China; the Guangzhou Army General Hospital, Guangzhou, China; the Third Affiliated Hospital of Sun Yat-sen University, Guangzhou, China; the First Affiliated Hospital of Sun Yat-sen University; Sun Yat-sen University Cancer Center; and the First Affiliated Hospital of Guangxi Medical University, Nanning, China) in three stages. All nodules in the training and total test set were pathologically confirmed. The diagnostic performance of ThyNet was first compared with 12 radiologists (test set A); a ThyNet-assisted strategy, in which ThyNet assisted diagnoses made by radiologists, was developed to improve diagnostic performance of radiologists using images (test set B); the ThyNet assisted strategy was then tested in a real-world clinical setting (using images and videos; test set C). In a simulated scenario, the number of unnecessary fine needle aspirations avoided by ThyNet-assisted strategy was calculated. FINDINGS: The area under the receiver operating characteristic curve (AUROC) for accurate diagnosis of ThyNet (0·922 [95% CI 0·910-0·934]) was significantly higher than that of the radiologists (0·839 [0·834-0·844]; p<0·0001). Furthermore, ThyNet-assisted strategy improved the pooled AUROC of the radiologists from 0·837 (0·832-0·842) when diagnosing without ThyNet to 0·875 (0·871-0·880; p<0·0001) with ThyNet for reviewing images, and from 0·862 (0·851-0·872) to 0·873 (0·863-0·883; p<0·0001) in the clinical test, which used images and videos. In the simulated scenario, the number of fine needle aspirations decreased from 61·9% to 35·2% using the ThyNet-assisted strategy, while missed malignancy decreased from 18·9% to 17·0%. INTERPRETATION: The ThyNet-assisted strategy can significantly improve the diagnostic performance of radiologists and help reduce unnecessary fine needle aspirations for thyroid nodules. FUNDING: National Natural Science Foundation of China and Guangzhou Science and Technology Project.
Authors: Daniel L Rubin; Terry S Desser; Rikiya Yamashita; Tara Kapoor; Minhaj Nur Alam; Alfiia Galimzianova; Saad Ali Syed; Mete Ugur Akdogan; Emel Alkim; Andrew Louis Wentland; Nikhil Madhuripan; Daniel Goff; Victoria Barbee; Natasha Diba Sheybani; Hersh Sagreiya Journal: Radiol Artif Intell Date: 2022-05-11
Authors: Sebastian Ziegelmayer; Markus Graf; Marcus Makowski; Joshua Gawlitza; Felix Gassert Journal: Cancers (Basel) Date: 2022-03-29 Impact factor: 6.639
Authors: Gabriela Maria Berinde; Andreea Iulia Socaciu; Mihai Adrian Socaciu; Andreea Cozma; Armand Gabriel Rajnoveanu; Gabriel Emil Petre; Doina Piciu Journal: Diagnostics (Basel) Date: 2022-01-27
Authors: Eoin F Cleere; Matthew G Davey; Shane O'Neill; Mel Corbett; John P O'Donnell; Sean Hacking; Ivan J Keogh; Aoife J Lowery; Michael J Kerin Journal: Diagnostics (Basel) Date: 2022-03-24