Andrew J Buckler1, Jovanna Danagoulian2, Kjell Johnson3, Adele Peskin4, Marios A Gavrielides5, Nicholas Petrick5, Nancy A Obuchowski6, Hubert Beaumont7, Lubomir Hadjiiski8, Rudresh Jarecha9, Jan-Martin Kuhnigk10, Ninad Mantri11, Michael McNitt-Gray12, Jan H Moltz10, Gergely Nyiri13, Sam Peterson14, Pierre Tervé15, Christian Tietjen16, Etienne von Lavante17, Xiaonan Ma2, Samantha St Pierre2, Maria Athelogou18. 1. Elucid Bioimaging Inc., 225 Main Street, Wenham, MA 01984. Electronic address: andrew.buckler@elucidbio.com. 2. Elucid Bioimaging Inc., 225 Main Street, Wenham, MA 01984. 3. Arbor Analytics LLC, Ann Arbor, Michigan. 4. National Institute of Standards and Technology, Boulder, Colorado. 5. U.S. Food and Drug Administration, Silver Spring, Maryland. 6. Cleveland Clinic, Cleveland, Ohio. 7. MEDIAN Technologies, Valbonne, France. 8. Department of Radiology, University of Michigan, Ann Arbor, Michigan. 9. Perceptive Informatics, Sundew Properties SEZ Pvt Ltd Mindspace, Hyderabad, Andhra Pradesh, India. 10. Fraunhofer MEVIS, Institute for Medical Image Computing, Bremen, Germany. 11. ICON Medical Imaging, Warrington, Pennsylvania. 12. Department of Radiology, University of California at Los Angeles, Los Angeles, California. 13. GE Healthcare, Buc, France. 14. Vital Images, Inc., Los Angeles, California. 15. KEOSYS, Saint-Herblain, France. 16. Siemens AG, Healthcare Sector, Imaging and Therapy Division, Forchheim, Germany. 17. Mirada Medical Ltd., Oxford Center for Innovation, Oxford, United Kingdom. 18. Definiens AG, München, Germany.
Abstract
RATIONALE AND OBJECTIVES: Tumor volume change has potential as a biomarker for diagnosis, therapy planning, and treatment response. Precision was evaluated and compared among semiautomated lung tumor volume measurement algorithms from clinical thoracic computed tomography data sets. The results inform approaches and testing requirements for establishing conformance with the Quantitative Imaging Biomarker Alliance (QIBA) Computed Tomography Volumetry Profile. MATERIALS AND METHODS: Industry and academic groups participated in a challenge study. Intra-algorithm repeatability and inter-algorithm reproducibility were estimated. Relative magnitudes of various sources of variability were estimated using a linear mixed effects model. Segmentation boundaries were compared to provide a basis on which to optimize algorithm performance for developers. RESULTS: Intra-algorithm repeatability ranged from 13% (best performing) to 100% (least performing), with most algorithms demonstrating improved repeatability as the tumor size increased. Inter-algorithm reproducibility was determined in three partitions and was found to be 58% for the four best performing groups, 70% for the set of groups meeting repeatability requirements, and 84% when all groups but the least performer were included. The best performing partition performed markedly better on tumors with equivalent diameters greater than 40 mm. Larger tumors benefitted by human editing but smaller tumors did not. One-fifth to one-half of the total variability came from sources independent of the algorithms. Segmentation boundaries differed substantially, not ony in overall volume but also in detail. CONCLUSIONS: Nine of the 12 participating algorithms pass precision requirements similar to what is indicated in the QIBA Profile, with the caveat that the present study was not designed to explicitly evaluate algorithm profile conformance. Change in tumor volume can be measured with confidence to within ±14% using any of these nine algorithms on tumor sizes greater than 10 mm. No partition of the algorithms was able to meet the QIBA requirements for interchangeability down to 10 mm, although the partition comprising best performing algorithms did meet this requirement for a tumor size of greater than approximately 40 mm.
RATIONALE AND OBJECTIVES:Tumor volume change has potential as a biomarker for diagnosis, therapy planning, and treatment response. Precision was evaluated and compared among semiautomated lung tumor volume measurement algorithms from clinical thoracic computed tomography data sets. The results inform approaches and testing requirements for establishing conformance with the Quantitative Imaging Biomarker Alliance (QIBA) Computed Tomography Volumetry Profile. MATERIALS AND METHODS: Industry and academic groups participated in a challenge study. Intra-algorithm repeatability and inter-algorithm reproducibility were estimated. Relative magnitudes of various sources of variability were estimated using a linear mixed effects model. Segmentation boundaries were compared to provide a basis on which to optimize algorithm performance for developers. RESULTS: Intra-algorithm repeatability ranged from 13% (best performing) to 100% (least performing), with most algorithms demonstrating improved repeatability as the tumor size increased. Inter-algorithm reproducibility was determined in three partitions and was found to be 58% for the four best performing groups, 70% for the set of groups meeting repeatability requirements, and 84% when all groups but the least performer were included. The best performing partition performed markedly better on tumors with equivalent diameters greater than 40 mm. Larger tumors benefitted by human editing but smaller tumors did not. One-fifth to one-half of the total variability came from sources independent of the algorithms. Segmentation boundaries differed substantially, not ony in overall volume but also in detail. CONCLUSIONS: Nine of the 12 participating algorithms pass precision requirements similar to what is indicated in the QIBA Profile, with the caveat that the present study was not designed to explicitly evaluate algorithm profile conformance. Change in tumor volume can be measured with confidence to within ±14% using any of these nine algorithms on tumor sizes greater than 10 mm. No partition of the algorithms was able to meet the QIBA requirements for interchangeability down to 10 mm, although the partition comprising best performing algorithms did meet this requirement for a tumor size of greater than approximately 40 mm.
Authors: Marios A Gavrielides; Rongping Zeng; Lisa M Kinnard; Kyle J Myers; Nicholas Petrick Journal: IEEE Trans Med Imaging Date: 2010-06-17 Impact factor: 10.048
Authors: H Bolte; C Riedel; C Riede; S Müller-Hülsbeck; S Freitag-Wolf; G Kohl; T Drews; M Heller; J Biederer; J Bieder Journal: Br J Radiol Date: 2007-06 Impact factor: 3.039
Authors: Sebastian Keil; Cedric Plumhans; Florian F Behrendt; Sven Stanzel; Michael Suehling; Georg Mühlenbruch; Andreas H Mahnken; Rolf W Günther; Marco Das Journal: Invest Radiol Date: 2009-02 Impact factor: 6.016
Authors: Michael F McNitt-Gray; Luc M Bidaut; Samuel G Armato; Charles R Meyer; Marios A Gavrielides; Charles Fenimore; Geoffrey McLennan; Nicholas Petrick; Binsheng Zhao; Anthony P Reeves; Reinhard Beichel; Hyun-Jung Grace Kim; Lisa Kinnard Journal: Transl Oncol Date: 2009-12 Impact factor: 4.243
Authors: Wenli Cai; Seth M Steinberg; Miriam A Bredella; Gina Basinsky; Bhanusupriya Somarouthu; Scott R Plotkin; Jeffrey Solomon; Brigitte C Widemann; Gordon J Harris; Eva Dombi Journal: Acad Radiol Date: 2017-10-31 Impact factor: 3.173
Authors: Marios A Gavrielides; Benjamin P Berman; Mark Supanich; Kurt Schultz; Qin Li; Nicholas Petrick; Rongping Zeng; Jenifer Siegelman Journal: Quant Imaging Med Surg Date: 2017-12