Benjamin E Nelms1, Greg Robinson2, Jay Markham2, Kyle Velasco2, Steve Boyd2, Sharath Narayan2, James Wheeler3, Mark L Sobczak4. 1. Canis Lupus LLC, Merrimac, Wisconsin; Department of Human Oncology, University of Wisconsin, Madison, Wisconsin. Electronic address: alpha@canislupusllc.com. 2. Radiation Oncology Resources, Goshen, Indiana. 3. Department of Radiation Oncology, Goshen Health System, Goshen, Indiana. 4. Fox Chase Cancer Center, Philadelphia, Pennsylvania.
Abstract
PURPOSE: This study quantifies variation in radiation treatment plan quality for plans generated by a population of treatment planners given very specific plan objectives. METHODS AND MATERIALS: A "Plan Quality Metric" (PQM) with 14 submetrics, each with a unique value function, was defined for a prostate treatment plan, serving as specific goals of a hypothetical "virtual physician." The exact PQM logic was distributed to a population of treatment planners (to remove ambiguity of plan goals or plan assessment methodology) as was a predefined computed tomographic image set and anatomic structure set (to remove anatomy delineation as a variable). Treatment planners used their clinical treatment planning system (TPS) to generate their best plan based on the specified goals and submitted their results for analysis. RESULTS: One hundred forty datasets were received and 125 plans accepted and analyzed. There was wide variability in treatment plan quality (defined as the ability of the planners and plans to meet the specified goals) quantified by the PQM. Despite the variability, the resulting PQM distributions showed no statistically significant difference between TPS employed, modality (intensity modulated radiation therapy versus arc), or education and certification status of the planner. The PQM results showed negligible correlation to number of beam angles, total monitor units, years of experience of the planner, or planner confidence. CONCLUSIONS: The ability of the treatment planners to meet the specified plan objectives (as quantified by the PQM) exhibited no statistical dependence on technologic parameters (TPS, modality, plan complexity), nor was the plan quality statistically different based on planner demographics (years of experience, confidence, certification, and education). Therefore, the wide variation in plan quality could be attributed to a general "planner skill" category that would lend itself to processes of continual improvement where best practices could be derived and disseminated to improve the mean quality and minimize the variation in any population of treatment planners.
PURPOSE: This study quantifies variation in radiation treatment plan quality for plans generated by a population of treatment planners given very specific plan objectives. METHODS AND MATERIALS: A "Plan Quality Metric" (PQM) with 14 submetrics, each with a unique value function, was defined for a prostate treatment plan, serving as specific goals of a hypothetical "virtual physician." The exact PQM logic was distributed to a population of treatment planners (to remove ambiguity of plan goals or plan assessment methodology) as was a predefined computed tomographic image set and anatomic structure set (to remove anatomy delineation as a variable). Treatment planners used their clinical treatment planning system (TPS) to generate their best plan based on the specified goals and submitted their results for analysis. RESULTS: One hundred forty datasets were received and 125 plans accepted and analyzed. There was wide variability in treatment plan quality (defined as the ability of the planners and plans to meet the specified goals) quantified by the PQM. Despite the variability, the resulting PQM distributions showed no statistically significant difference between TPS employed, modality (intensity modulated radiation therapy versus arc), or education and certification status of the planner. The PQM results showed negligible correlation to number of beam angles, total monitor units, years of experience of the planner, or planner confidence. CONCLUSIONS: The ability of the treatment planners to meet the specified plan objectives (as quantified by the PQM) exhibited no statistical dependence on technologic parameters (TPS, modality, plan complexity), nor was the plan quality statistically different based on planner demographics (years of experience, confidence, certification, and education). Therefore, the wide variation in plan quality could be attributed to a general "planner skill" category that would lend itself to processes of continual improvement where best practices could be derived and disseminated to improve the mean quality and minimize the variation in any population of treatment planners.
Authors: Kevin L Moore; Rachel Schmidt; Vitali Moiseenko; Lindsey A Olsen; Jun Tan; Ying Xiao; James Galvin; Stephanie Pugh; Michael J Seider; Adam P Dicker; Walter Bosch; Jeff Michalski; Sasa Mutic Journal: Int J Radiat Oncol Biol Phys Date: 2015-04-03 Impact factor: 7.038
Authors: J M A M Kusters; K Bzdusek; P Kumar; P G M van Kollenburg; M C Kunze-Busch; M Wendling; T Dijkema; J H A M Kaanders Journal: Strahlenther Onkol Date: 2017-08-02 Impact factor: 3.621
Authors: Conor K McGarry; Christina E Agnew; Mohammad Hussein; Yatman Tsang; Alan McWilliam; Alan R Hounsell; Catharine H Clark Journal: Br J Radiol Date: 2015-10-29 Impact factor: 3.039
Authors: Nan Li; Ruben Carmona; Igor Sirak; Linda Kasaova; David Followill; Jeff Michalski; Walter Bosch; William Straube; Loren K Mell; Kevin L Moore Journal: Int J Radiat Oncol Biol Phys Date: 2016-10-13 Impact factor: 7.038