Abraham George Smith1,2, Jens Petersen1,2, Cynthia Terrones-Campos2,3, Anne Kiil Berthelsen2,4, Nora Jarrett Forbes1,2, Sune Darkner1, Lena Specht2, Ivan Richter Vogelius2,5. 1. Department of Computer Science, University of Copenhagen, Copenhagen, Denmark. 2. Department of Oncology, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark. 3. Department of Infectious Diseases, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark. 4. Department of Clinical Physiology, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark. 5. Department of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark.
Abstract
PURPOSE: Organ-at-risk contouring is still a bottleneck in radiotherapy, with many deep learning methods falling short of promised results when evaluated on clinical data. We investigate the accuracy and time-savings resulting from the use of an interactive-machine-learning method for an organ-at-risk contouring task. METHODS: We implement an open-source interactive-machine-learning software application that facilitates corrective-annotation for deep-learning generated contours on X-ray CT images. A trained-physician contoured 933 hearts using our software by delineating the first image, starting model training, and then correcting the model predictions for all subsequent images. These corrections were added into the training data, which was used for continuously training the assisting model. From the 933 hearts, the same physician also contoured the first 10 and last 10 in Eclipse (Varian) to enable comparison in terms of accuracy and duration. RESULTS: We find strong agreement with manual delineations, with a dice score of 0.95. The annotations created using corrective-annotation also take less time to create as more images are annotated, resulting in substantial time savings compared to manual methods. After 923 images had been delineated, hearts took 2 min and 2 s to delineate on average, which includes time to evaluate the initial model prediction and assign the needed corrections, compared to 7 min and 1 s when delineating manually. CONCLUSIONS: Our experiment demonstrates that interactive-machine-learning with corrective-annotation provides a fast and accessible way for non computer-scientists to train deep-learning models to segment their own structures of interest as part of routine clinical workflows.
PURPOSE: Organ-at-risk contouring is still a bottleneck in radiotherapy, with many deep learning methods falling short of promised results when evaluated on clinical data. We investigate the accuracy and time-savings resulting from the use of an interactive-machine-learning method for an organ-at-risk contouring task. METHODS: We implement an open-source interactive-machine-learning software application that facilitates corrective-annotation for deep-learning generated contours on X-ray CT images. A trained-physician contoured 933 hearts using our software by delineating the first image, starting model training, and then correcting the model predictions for all subsequent images. These corrections were added into the training data, which was used for continuously training the assisting model. From the 933 hearts, the same physician also contoured the first 10 and last 10 in Eclipse (Varian) to enable comparison in terms of accuracy and duration. RESULTS: We find strong agreement with manual delineations, with a dice score of 0.95. The annotations created using corrective-annotation also take less time to create as more images are annotated, resulting in substantial time savings compared to manual methods. After 923 images had been delineated, hearts took 2 min and 2 s to delineate on average, which includes time to evaluate the initial model prediction and assign the needed corrections, compared to 7 min and 1 s when delineating manually. CONCLUSIONS: Our experiment demonstrates that interactive-machine-learning with corrective-annotation provides a fast and accessible way for non computer-scientists to train deep-learning models to segment their own structures of interest as part of routine clinical workflows.
Authors: Marcus Griffiths; Benjamin M Delory; Vanessica Jawahir; Kong M Wong; G Cody Bagnall; Tyler G Dowd; Dmitri A Nusinow; Allison J Miller; Christopher N Topp Journal: Plant Cell Environ Date: 2022-01-24 Impact factor: 7.947