Maria Athelogou1, Hyun J Kim2, Alden Dima3, Nancy Obuchowski4, Adele Peskin3, Marios A Gavrielides5, Nicholas Petrick5, Ganesh Saiprasad3, Dirk Colditz Colditz6, Hubert Beaumont7, Estanislao Oubel7, Yongqiang Tan8, Binsheng Zhao8, Jan-Martin Kuhnigk9, Jan Hendrik Moltz9, Guillaume Orieux10, Robert J Gillies11, Yuhua Gu11, Ninad Mantri12, Gregory Goldmacher12, Luduan Zhang13, Emilio Vega14, Michael Bloom14, Rudresh Jarecha15, Grzegorz Soza16, Christian Tietjen16, Tomoyuki Takeguchi17, Hitoshi Yamagata18, Sam Peterson19, Osama Masoud19, Andrew J Buckler20. 1. Definiens AG, Bernhard-Wicki Str 5, 80636 Munich, Germany. Electronic address: mathelogou@definiens.com. 2. UCLA, Center for Computer Vision and Imaging Biomarkers, Dept. of Radiological Sciences David Geffen School of Medicine at UCLA Dept. of Biostatistics Fielding School of Public at UCLA, Los Angeles, USA. 3. National Institute of Standards and Technology, Gaithersburg, USA. 4. Quantitative Health Sciences/JJN3, Cleveland Clinic Foundation, Cleveland, USA. 5. U.S. Food and Drug Administration, Silver Spring, Maryland. 6. Consultant QM/RA, Jena, Germany. 7. MEDIAN Technologies, Valbonne Sophia Antipolis, France. 8. Columbia University Medical Center, Department of Radiology, New York, USA. 9. Fraunhofer MEVIS, Institute for Medical Image Computing, Bremen, Germany. 10. GE Healthcare, Buc, France. 11. Moffitt Cancer Center and Research Institute, Tampa, Florida, USA. 12. ICON Medical Imaging, Warrington, Pennsylvania, USA. 13. INTIO, Inc., Broomfield, Colorado, USA. 14. NYU Langone Medical Center Faculty Practice Radiology, New York, USA. 15. Perceptive Informatics, Andhra Pradesh, India. 16. Siemens AG, Healthcare Sector, Computed Tomography, Forchheim, Germany. 17. Toshiba Corporation, Corporate R&D Center, Kawasaki, Japan. 18. Toshiba Corporation, Toshiba Medical Systems Corporation, Otawara, Japan. 19. Vital Images, Inc. (a Toshiba Medical Systems Group), Minnesota, USA. 20. Buckler Biomedical Associates LLC, Massachusetts, USA.
Abstract
RATIONALE AND OBJECTIVES: Quantifying changes in lung tumor volume is important for diagnosis, therapy planning, and evaluation of response to therapy. The aim of this study was to assess the performance of multiple algorithms on a reference data set. The study was organized by the Quantitative Imaging Biomarker Alliance (QIBA). MATERIALS AND METHODS: The study was organized as a public challenge. Computed tomography scans of synthetic lung tumors in an anthropomorphic phantom were acquired by the Food and Drug Administration. Tumors varied in size, shape, and radiodensity. Participants applied their own semi-automated volume estimation algorithms that either did not allow or allowed post-segmentation correction (type 1 or 2, respectively). Statistical analysis of accuracy (percent bias) and precision (repeatability and reproducibility) was conducted across algorithms, as well as across nodule characteristics, slice thickness, and algorithm type. RESULTS: Eighty-four percent of volume measurements of QIBA-compliant tumors were within 15% of the true volume, ranging from 66% to 93% across algorithms, compared to 61% of volume measurements for all tumors (ranging from 37% to 84%). Algorithm type did not affect bias substantially; however, it was an important factor in measurement precision. Algorithm precision was notably better as tumor size increased, worse for irregularly shaped tumors, and on the average better for type 1 algorithms. Over all nodules meeting the QIBA Profile, precision, as measured by the repeatability coefficient, was 9.0% compared to 18.4% overall. CONCLUSION: The results achieved in this study, using a heterogeneous set of measurement algorithms, support QIBA quantitative performance claims in terms of volume measurement repeatability for nodules meeting the QIBA Profile criteria.
RATIONALE AND OBJECTIVES: Quantifying changes in lung tumor volume is important for diagnosis, therapy planning, and evaluation of response to therapy. The aim of this study was to assess the performance of multiple algorithms on a reference data set. The study was organized by the Quantitative Imaging Biomarker Alliance (QIBA). MATERIALS AND METHODS: The study was organized as a public challenge. Computed tomography scans of synthetic lung tumors in an anthropomorphic phantom were acquired by the Food and Drug Administration. Tumors varied in size, shape, and radiodensity. Participants applied their own semi-automated volume estimation algorithms that either did not allow or allowed post-segmentation correction (type 1 or 2, respectively). Statistical analysis of accuracy (percent bias) and precision (repeatability and reproducibility) was conducted across algorithms, as well as across nodule characteristics, slice thickness, and algorithm type. RESULTS: Eighty-four percent of volume measurements of QIBA-compliant tumors were within 15% of the true volume, ranging from 66% to 93% across algorithms, compared to 61% of volume measurements for all tumors (ranging from 37% to 84%). Algorithm type did not affect bias substantially; however, it was an important factor in measurement precision. Algorithm precision was notably better as tumor size increased, worse for irregularly shaped tumors, and on the average better for type 1 algorithms. Over all nodules meeting the QIBA Profile, precision, as measured by the repeatability coefficient, was 9.0% compared to 18.4% overall. CONCLUSION: The results achieved in this study, using a heterogeneous set of measurement algorithms, support QIBA quantitative performance claims in terms of volume measurement repeatability for nodules meeting the QIBA Profile criteria.
Authors: P Therasse; S G Arbuck; E A Eisenhauer; J Wanders; R S Kaplan; L Rubinstein; J Verweij; M Van Glabbeke; A T van Oosterom; M C Christian; S G Gwyther Journal: J Natl Cancer Inst Date: 2000-02-02 Impact factor: 13.506
Authors: Naama R Bogot; Ella A Kazerooni; Aine M Kelly; Leslie E Quint; Benoit Desjardins; Bin Nan Journal: Acad Radiol Date: 2005-08 Impact factor: 3.173
Authors: Chikako Suzuki; Hans Jacobsson; Thomas Hatschek; Michael R Torkzad; Katarina Bodén; Yvonne Eriksson-Alm; Elisabeth Berg; Hirofumi Fujii; Atsushi Kubo; Lennart Blomqvist Journal: Radiographics Date: 2008 Mar-Apr Impact factor: 5.333
Authors: Jeremy J Erasmus; Gregory W Gladish; Lyle Broemeling; Bradley S Sabloff; Mylene T Truong; Roy S Herbst; Reginald F Munden Journal: J Clin Oncol Date: 2003-07-01 Impact factor: 44.544
Authors: Nancy A Obuchowski; Anthony P Reeves; Erich P Huang; Xiao-Feng Wang; Andrew J Buckler; Hyun J Grace Kim; Huiman X Barnhart; Edward F Jackson; Maryellen L Giger; Gene Pennello; Alicia Y Toledano; Jayashree Kalpathy-Cramer; Tatiyana V Apanasovich; Paul E Kinahan; Kyle J Myers; Dmitry B Goldgof; Daniel P Barboriak; Robert J Gillies; Lawrence H Schwartz; Daniel C Sullivan Journal: Stat Methods Med Res Date: 2014-06-11 Impact factor: 3.021
Authors: Marios A Gavrielides; Lisa M Kinnard; Kyle J Myers; Jennifer Peregoy; William F Pritchard; Rongping Zeng; Juan Esparza; John Karanian; Nicholas Petrick Journal: Opt Express Date: 2010-07-05 Impact factor: 3.894
Authors: Yoganand Balagurunathan; Andrew Beers; Jayashree Kalpathy-Cramer; Michael McNitt-Gray; Lubomir Hadjiiski; Bensheng Zhao; Jiangguo Zhu; Hao Yang; Stephen S F Yip; Hugo J W L Aerts; Sandy Napel; Dmitrii Cherezov; Kenny Cha; Heang-Ping Chan; Carlos Flores; Alberto Garcia; Robert Gillies; Dmitry Goldgof Journal: Med Phys Date: 2018-02-19 Impact factor: 4.071
Authors: Kathryn E Keenan; Jana G Delfino; Kalina V Jordanova; Megan E Poorman; Prathyush Chirra; Akshay S Chaudhari; Bettina Baessler; Jessica Winfield; Satish E Viswanath; Nandita M deSouza Journal: Med Phys Date: 2021-09-29 Impact factor: 4.506
Authors: Marios A Gavrielides; Benjamin P Berman; Mark Supanich; Kurt Schultz; Qin Li; Nicholas Petrick; Rongping Zeng; Jenifer Siegelman Journal: Quant Imaging Med Surg Date: 2017-12