PURPOSE: Clinical data collection and development of outcome prediction models by machine learning can form the foundation for a learning health system offering precision radiation therapy. However, changes in clinical practice over time can affect the measures and patient outcomes and, hence, the collected data. We hypothesize that regular prediction model updates and continuous prospective data collection are important to prevent the degradation of a model's predication accuracy. METHODS AND MATERIALS: Clinical and dosimetric data from head and neck patients receiving intensity modulated radiation therapy from 2008 to 2015 were prospectively collected as a routine clinical workflow and anonymized for this analysis. Prediction models for grade ≥2 xerostomia at 3 to 6 months of follow-up were developed by bivariate logistic regression using the dose-volume histogram of parotid and submandibular glands. A baseline prediction model was developed with a training data set from 2008 to 2009. The selected predictor variables and coefficients were updated by 4 different model updating methods. (A) The prediction model was updated by using only recent 2-year data and applied to patients in the following test year. (B) The model was updated by increasing the training data set yearly. (C) The model was updated by increasing the training data set on the condition that the area under the curve (AUC) of the recent test year was less than 0.6. (D) The model was not updated. The AUC of the test data set was compared among the 4 model updating methods. RESULTS: Dose to parotid and submandibular glands and grade of xerostomia showed decreasing trends over the years (2008-2015, 297 patients; P < .001). The AUC of predicting grade ≥2 xerostomia for the initial training data set (2008-2009, 41 patients) was 0.6196. The AUC for the test data set (2010-2015, 256 patients) decreased to 0.5284 when the initial model was not updated (D). However, the AUC was significantly improved by model updates (A: 0.6164; B: 0.6084; P < .05). When the model was conditionally updated, the AUC was 0.6072 (C). CONCLUSIONS: Our preliminary results demonstrate that updating prediction models with prospective data collection is effective for maintaining the performance of xerostomia prediction. This suggests that a machine learning framework can handle the dynamic changes in a radiation oncology clinical practice and may be an important component for the construction of a learning health system.
PURPOSE: Clinical data collection and development of outcome prediction models by machine learning can form the foundation for a learning health system offering precision radiation therapy. However, changes in clinical practice over time can affect the measures and patient outcomes and, hence, the collected data. We hypothesize that regular prediction model updates and continuous prospective data collection are important to prevent the degradation of a model's predication accuracy. METHODS AND MATERIALS: Clinical and dosimetric data from head and neck patients receiving intensity modulated radiation therapy from 2008 to 2015 were prospectively collected as a routine clinical workflow and anonymized for this analysis. Prediction models for grade ≥2 xerostomia at 3 to 6 months of follow-up were developed by bivariate logistic regression using the dose-volume histogram of parotid and submandibular glands. A baseline prediction model was developed with a training data set from 2008 to 2009. The selected predictor variables and coefficients were updated by 4 different model updating methods. (A) The prediction model was updated by using only recent 2-year data and applied to patients in the following test year. (B) The model was updated by increasing the training data set yearly. (C) The model was updated by increasing the training data set on the condition that the area under the curve (AUC) of the recent test year was less than 0.6. (D) The model was not updated. The AUC of the test data set was compared among the 4 model updating methods. RESULTS: Dose to parotid and submandibular glands and grade of xerostomia showed decreasing trends over the years (2008-2015, 297 patients; P < .001). The AUC of predicting grade ≥2 xerostomia for the initial training data set (2008-2009, 41 patients) was 0.6196. The AUC for the test data set (2010-2015, 256 patients) decreased to 0.5284 when the initial model was not updated (D). However, the AUC was significantly improved by model updates (A: 0.6164; B: 0.6084; P < .05). When the model was conditionally updated, the AUC was 0.6072 (C). CONCLUSIONS: Our preliminary results demonstrate that updating prediction models with prospective data collection is effective for maintaining the performance of xerostomia prediction. This suggests that a machine learning framework can handle the dynamic changes in a radiation oncology clinical practice and may be an important component for the construction of a learning health system.
Authors: Anshu K Jain; Sanjay Aneja; Clifton D Fuller; Adam P Dicker; Caroline Chung; Erika Kim; Justin S Kirby; Harry Quon; Clara J K Lam; William C Louv; Chris Ahern; Ying Xiao; Todd R McNutt; Nadine Housri; Ronald D Ennis; John Kang; Ying Tang; Howard Higley; Michelle A Berny-Lang; Kevin A Camphausen Journal: JCO Clin Cancer Inform Date: 2020-08
Authors: Eva Onjukka; Claes Mercke; Einar Björgvinsson; Anna Embring; Anders Berglund; Gabriella Alexandersson von Döbeln; Signe Friesland; Giovanna Gagliardi; Clara Lenneby Helleday; Helena Sjödin; Ingmar Lax Journal: Front Oncol Date: 2020-08-14 Impact factor: 6.244
Authors: David A Jones; Joel Smith; Xue W Mei; Maria A Hawkins; Tim Maughan; Frank van den Heuvel; Thomas Mee; Karen Kirkby; Norman Kirkby; Alastair Gray Journal: Clin Transl Radiat Oncol Date: 2019-10-31