Evaluation of classification and regression tree (CART) model in weight loss prediction following head and neck cancer radiation therapy.

OBJECTIVE: We explore whether a knowledge-discovery approach building a Classification and Regression Tree (CART) prediction model for weight loss (WL) in head and neck cancer (HNC) patients treated with radiation therapy (RT) is feasible. METHODS AND MATERIALS: HNC patients from 2007 to 2015 were identified from a prospectively collected database Oncospace. Two prediction models at different time points were developed to predict weight loss ≥5 kg at 3 months post-RT by CART algorithm: (1) during RT planning using patient demographic, delineated dose data, planning target volume-organs at risk shape relationships data and (2) at the end of treatment (EOT) using additional on-treatment toxicities and quality of life data.
RESULTS: Among 391 patients identified, WL predictors during RT planning were International Classification of Diseases diagnosis; dose to masticatory and superior constrictor muscles, larynx, and parotid; and age. At EOT, patient-reported oral intake, diagnosis, N stage, nausea, pain, dose to larynx, parotid, and low-dose planning target volume-larynx distance were significant predictive factors. The area under the curve during RT and EOT was 0.773 and 0.821, respectively.
CONCLUSIONS: We demonstrate the feasibility and potential value of an informatics infrastructure that has facilitated insight into the prediction of WL using the CART algorithm. The prediction accuracy significantly improved with the inclusion of additional treatment-related data and has the potential to be leveraged as a strategy to develop a learning health system.

This study evaluates the feasibility of developing the classification and regression tree model to predict weight loss following head and neck cancer radiation therapy. Two time-dependent models were built and showed improving area under the curve with incremental data (1) at the time of radiation therapy planning using demographic, dosimetric, and shape relationship and (2) at the end of treatment using additional toxicities and patient-reported outcomes among head and neck cancer patients.

Alt-text: Unlabelled box

Introduction

Precision care is an emerging approach for the prevention and treatment of disease states that accounts for individual heterogeneity in genes, environment, and life-style.1, 2, 3, 4, 5, 6, 7, 8 To accomplish the goal of precision radiation medicine, our group has leveraged an informatic infrastructure centered around our Oncospace database,9, 10 which contains data captured at the point of care (PoC) during the routine clinical care of cancer patients (Fig 1). Our overarching hypothesis is that this diverse and comprehensive clinical dataset growing within our informatics infrastructure can facilitate the requisite knowledge–discovery and development of machine learning–based predictive algorithms that offer particular advantages compared with traditional approaches to guide individualized patient decisions as a component of a learning health system.12, 13, 14

Figure 1

Timeline for patient care, data capture, and storage in the Oncospace database and weight loss prediction models (top). Data inventory with weight, toxicities, and patient-reported quality of life among head and neck cancer patients from 2008 through 2015 (bottom). RT, radiation therapy.

Weight loss is a common side effect among head and neck cancer (HNC) patients throughout treatment and follow-up (Fig 2) and results from multiple toxicities. During chemotherapy radiation therapy (CRT), critical weight loss has been reported to be as high as 57% in irradiated HNC patients. HNC patients are at risk of reduced oral intake because of CRT-induced side effects such as taste and/or smell alternations, painful mucosal inflammation of the oral cavity and pharynx, secretion changes (xerostomia or thick, ropey secretions), and swallow muscle damage caused by radiation injuries and/or surgical ablation of muscular and nervous structures. Moreover, these symptoms may become progressively worse during and after treatment, compromising quality of life (QoL) and contributing to poor treatment outcomes and prognosis.16, 18, 19 It is important for care providers to identify patients at risk of critical weight loss, with the goal of effective weight management or, ideally, the prophylactic prevention of malnutrition. Interventions include possible treatment plan modifications, feeding tube placement, pain management, swallow exercise, and frequent monitoring, which can be used during treatment or posttreatment follow-up if the patient is at risk for critical weight loss after treatment.

Figure 2

Trend of mean weight difference from the end of treatment (kilograms). Patients with and without critical weight loss (≥5 kg) at 3 months posttreatment are stratified.

The causal relationship between RT and weight loss is highly multifactorial. To identify patients at risk of critical weight loss, we applied an exploratory knowledge–discovery approach. Several studies have investigated the predictive risk factors of weight loss during or after the treatment, mainly using clinical factors.20, 21 In this study, we used all the dosimetric information available in an RT plan in addition to the clinical factors and piloted the use of the classification and regression tree (CART) algorithm to develop a predictive algorithm for critical weight loss in irradiated HNC patients.

Methods and materials

Database and data collection

Oncospace is an analytical database that has systematically aggregated prospectively collected clinical outcomes, RT plans, and geometric shape relationship at our institution since 2007 (Fig 1).6, 9 Clinical data such as patient demographic information, medical histories, physician-assessed toxicities, and patient-reported QoL were directly and routinely collected into the database at the PoC, facilitated by a Web interface to the database (Fig 1). Patients were treated with intensity modulated RT (IMRT) for 6 to 7 weeks with either a once- or twice-daily hyperfractionated schedule. Toxicity grading defined by the Common Terminology Criteria for Adverse Events (versions 3.0 and 4.0) was assessed during weekly on-treatment visits and follow-up visits; patient-reported outcomes (PROs) including QoL were measured by the Functional Assessment of Cancer Therapy-Head and Neck, and the MD Anderson Dysphagia Inventory instruments22, 23, 24, 25, 26 were assessed at the end of treatment (EOT) and follow-up visits. For RT, prescribed dose to the primary tumor volume and clinically involved nodes was 70 to 72 Gy with 2.0 to 2.2 Gy per fraction, combined with standard dose to moderate- (63 Gy) and low-risk (57 Gy) nodal volumes. Comprehensive dosimetric data defined as the dose delivered at 1% volume increments for each high-/mid-/low-risk planning target volume (PTV) and each organ at risk (OAR) were computed in Oncospace.

Study inclusion criteria were HNC patients treated with IMRT ± concurrent chemotherapy ± induction chemotherapy from 2007 through 2015 and patients with weight measurements both at the end of treatment and at 3 month follow-up. Patients with multiple treatments, reirradiation and palliative RT, and missing weight measurements at these time points were excluded.

Statistical design and analysis

Weight loss of ≥5 kg (yes/no) at 3 months (60-120 days) posttreatment from the EOT was the primary study endpoint (Fig 2). To potentiate decision support through the RT, 2 prediction models were developed: model 1 at the time of the RT planning with variables available before RT and model 2 at the EOT with additional clinical assessments variables captured at EOT. Model 1 used diagnosis as coded by the International Statistical Classification of Diseases and Related Health Problems (ICD-9) code, TNM staging, age, and dose-volume histogram data at 1% volume increments for OAR as follows: (1) muscles: inferior/middle/superior constrictors, cricopharyngeus, and masticatory (masticator, temporalis, medial, and lateral pterygoid); (2) swallow pathway: oral cavity, oral mucosa, soft palate, and larynx; and (3) secretion: parotid and submandibular glands. Lateralized OARs (eg, left and right parotid glands) were combined into a single structure for calculating dose-volume histogram. Minimum distances (centimeters) between high-/mid-/low-risk PTV and OARs were extracted from the records of geometric shape relationships calculated by the overlap volume histogram algorithm.27, 28, 29, 30 Model 2 included additional data such as toxicity and QoL. For knowledge–discovery purposes, QoL questions were treated as individual information and analyzed independently.

Baseline characteristics were compared with binary weight loss status (≥5 vs <5 kg) at 3 months posttreatment by t test and Wilcoxon test for continuous variables and χ2 test for categorical variables (Table 1). Statistical significance was defined as P < .05.

Table 1

Study population characteristics for patients by critical weight loss (<5 vs ≥5 kg) at RT planning and at the end of RT

Parameters	N	No CWL reference	CWL	P value
At RT planning: demographic	N = 391	N = 297	N = 94
Age, mean (SD)	57.39 (10.69)	56.92 (10.99)	58.84 (9.76)	.133
Sex, n (%)				.901
Man	306 (78.26)	232 (78.11)	74 (78.72)
Women	85 (21.74)	65 (21.89)	20 (21.28)
Race, n (%)				.111
Caucasian	187 (76.64)	128 (74.42)	59 (81.94)
African American	42 (17.21)	32 (18.60)	10 (13.89)
Asian	4 (1.64)	4 (2.33)	0 (0.00)
Others	10 (4.10)	8 (4.65)	2 (1.39)
Tumor site, n (%)				.040
Oral cavity	3 (0.77)	2 (0.67)	1 (1.06)
Nasopharynx	14 (3.58)	9 (3.03)	5 (5.32)
Oropharynx	157 (40.15)	108 (36.36)	49 (52.13)
Hypopharynx	9 (2.30)	7 (2.36)	2 (2.13)
Larynx	46 (11.76)	41 (13.80)	5 (5.32)
Other	162 (41.43)	130 (43.77)	32 (34.04)
Tumor classification, n (%)				.174
0	21 (7.22)	12 (5.45)	9 (12.68)
1	60 (20.62)	47 (21.36)	13 (18.31)
2	96 (32.99)	72 (32.73)	24 (33.80)
3	41 (14.09)	35 (15.91)	6 (8.45)
4	73 (25.09)	54 (24.55)	19 (26.76)
N classification, n (%)				.494
0	77 (26.55)	62 (28.31)	15 (21.13)
1	44 (15.17)	31 (14.16)	13 (18.31)
2	161 (55.52)	121 (55.25)	40 (56.34)
3	8 (2.76)	5 (2.28)	3 (4.23)
M classification, n (%)				.073
No	255 (98.84)	197 (99.49)	58 (96.67)
Yes	3 (1.16)	1 (0.51)	2 (3.33)
Overall stage, n (%)				.707
0	3 (1.22)	2 (1.06)	1 (1.75)
1	17 (6.94)	15 (7.98)	2 (3.51)
2	19 (7.76)	14 (7.45)	5 (8.77)
3	37 (15.10)	30 (15.96)	7 (12.28)
4	169 (68.98)	127 (67.55)	127 (73.68)
Treatment modality, n (%)				.240
RT alone	127 (32.48)	99 (33.33)	28 (29.79)
CRT	253 (64.71)	191 (64.31)	62 (65.96)
CRT + surgery	8 (2.05)	4 (1.35)	4 (4.26)
Surgery + RT	3 (0.77)	3 (1.01)	0 (0.00)
Smoking status, n (%)				.807
Never smoked	67 (40.36)	58 (41.13)	9 (36.00)
Quit smoking	90 (54.22)	75 (53.19)	15 (60.00)
Currently smoking	9 (5.42)	8 (5.67)	1 (4.00)
HPV, n (%)				.810
Yes	56 (58.95)	47 (59.49)	9 (56.25)
No	39 (41.05)	32 (40.51)	7 (43.75)
At RT planning: dosimetric
PTV D95 dose, n (%)				.193
≤65 Gy	89 (23.73)	72 (25.35)	17 (18.68)
>65 Gy	286 (76.27)	212 (74.65)	74 (81.31)
Combined parotid D95, mean (SD)	287	8.61 (5.89)	10.85 (6.14)	.002
Combined parotid D50, mean (SD)	287	25.61 (13.45)	28.98 (12.23)	.021
Combined submandibular D95, mean (SD)	232	42.25 (19.97)	47.77 (17.06)	.044
Combined submandibular D50, mean (SD)	232	54.08 (19.85)	58.51 (15.92)	.206
Larynx D95, mean (SD)	258	27.56 (17.2)	31.37 (16.21)	.248
Larynx D50, mean (SD)	258	37.61 (17.05)	43.21 (14.92)	.339
Combined masticatory muscle D95, mean (SD)	150	11.98 (10.7)	17.39 (10.22)	.002
Combined masticatory muscle D50, mean (SD)	150	33.71 (16.53)	41.5 (13.67)	.002
Superior constrictor muscle D95, mean (SD)	149	47.81 (17.21)	53.03 (14.35)	.046
Superior constrictor muscle D50, mean (SD)	149	56.66 (14.88)	60.05 (13.82)	.044
Middle constrictor muscle D95, mean (SD)	142	45.32 (16.36)	47.84 (12.67)	.634
Middle constrictor muscle D50, mean (SD)	142	53.65 (15.07)	56.35 (12.23)	.397
Inferior constrictor muscle D95, mean (SD)	141	36.78 (16.15)	40.45 (12.6)	.188
Inferior constrictor muscle D50, mean (SD)	141	43.91 (15.99)	46.36 (12.24)	.460
Pharyngeal constrictor muscle D95, mean (SD)	75	22.32 (15.73)	24.13 (15.44)	.631
Pharyngeal constrictor muscle D50, mean (SD)	75	40.04 (16.65)	35.18 (20.41)	.596
Cricopharyngeal muscle D95, mean (SD)	107	34.27 (14.67)	40.93 (10.77)	.043
Cricopharyngeal muscle D50, mean (SD)	107	38.34 (15.48)	44.27 (10.3)	.121
At the end of RT: additional assessments
Karnofsky performance status, n (%)
≥80	282 (74.80)	215 (75.44)	67 (72.83)	.616
<80	95 (25.20)	70 (24.56)	25 (27.17)
Pain at EOT, n (%)				.317
≥5	109 (27.88)	79 (26.00)	30 (31.91)
<5	282 (72.12)	218 (73.40)	64 (68.09)
Dysphagia at EOT, mean (SD)	332	0.72 (0.99)	0.85 (0.98)	.318
Xerostomia at EOT, mean (SD)	391	1.41 (0.75)	1.59 (0.63)	.030
Food oral intake score, n (%)				.056
≥4	113 (84.33)	99 (86.84)	14 (70.00)
<4	21 (15.67)	15 (13.16)	6 (30.00)
Penetration Aspiration Scale, n (%)				.711
≤2	66 (70.21)	52 (69.33)	14 (73.68)
>2	28 (29.79)	23 (30.67)	5 (26.32)
PEG/NG tube used, n (%)				.134
Yes	122 (75.31)	86 (72.27)	36 (83.72)
No	40 (24.69)	33 (27.73)	7 (17.50)

CRT, chemotherapy radiation therapy; CWL, critical weight loss; D95, dose to 95% of muscle; EOT, end of treatment; HPV, human papilloma virus; NG, nasogastric; PEG, percutaneous endoscopic gastrostomy; PTV, planning target volume; RT, radiation therapy; SD, standard deviation.

Decision tree models predicting weight loss were built with the CART algorithm. The cohort was partitioned into 2 subsets based on a predictor variable with the highest risk of weight loss. This process was repeated on each derived subset in an iterative manner (ie, recursive partitioning). At each partition, the information gain criteria were used to determine the partitioning predictor variable with a binary threshold for continuous or ordinal variables and with binary partitioning.

To handle the missing values in each primary predictor, surrogate variables were used to split the observations missing the primary predictor. A maximum of 5 surrogates were found for each primary predictor by reapplying the partitioning algorithm (without recursion) to estimate the binary split of the primary predictor. If all 5 surrogates were missing, then the observation was sent in the majority direction.32, 33 Leave-one-out cross-validation was used to determine the number of predictor variables selected in the decision tree models and to calculate the model performance. The DeLong test was further used to compare the area under the curve of the 2 models. All the statistical analysis were performed by R 3.2.2 with the rpart package.

Results

A total of 391 patients met the study inclusion criteria. The study population consisted of more men (n = 306, 78%) and early stage (T ≤ 2) cancer (n = 187, 75%). The mean age was 57 years, and the mean weight loss at 3 months posttreatment was 2.4 kg. Nearly one-half of the study population had tumors located in the pharynx (nasopharynx, oropharynx, and hypopharynx) (n = 180, 46%) (Table 1).

Predictors for critical weight loss

Patient lost weight throughout radiation treatment. Mean weight did not decrease further after the end of the treatment, and started to increase at the 3-month follow-up (Fig 2). At the 3-month follow-up, 94 (24%) patients continued to lose weight beyond the 5 kg threshold. Analyses using a 5% weight loss threshold are presented in eTable 1 (available as supplementary material online only at www.practical.radonc.org).

Model 1 was developed at the planning phase of RT when dosimetric and tumor-related variables were available (Fig 3). Anatomic tumor site by ICD-9 code was particularly important. Dose to 90% of the masticatory muscle (D90) with threshold of 14 Gy, D100 (40 Gy) of superior constrictor muscle, D78 (24 Gy) of larynx, and D89 (15 Gy) of parotid glands were also selected from RT and dosimetric variables. Age (58 years) was the last predictor of critical weight loss. The area under the curve, sensitivity, specificity, and positive predictive and negative predictive values by cross-validation were 0.77, 0.77, 0.67, 0.43, and 0.90, respectively (Table 2).

Figure 3

Prediction tree during planning phase (model 1) for critical (≥5 kg) WL among head and neck cancer patients 3 months posttreatment. Nodes display sample size (n) and the percentage of patients with critical weight loss within the (sub)group. The potential risk factors in the CART analysis were ICD-9 code; dose to masticatory muscle, superior constrictor muscle, larynx, and parotid; and age. CART, classification and regression tree; D100, dose to 100% of organ; ICD-9, International Classification of Diseases-9; WL, weight loss.

Table 2

Characteristics of the weight loss prediction models (<5 vs ≥5 kg) at planning and end of the RT treatment

Parameters	At RT planning	At the end of RT
Area under the curve	0.77	0.82a
Sensitivity	0.77	0.98
Specificity	0.67	0.59
Positive predictive value	0.43	0.46
Negative predictive value	0.90	0.99

Abbreviation as in Table 1.

Statistical significance at RT planning and at the end of RT.

To interpret model 1, the principal discriminating node was tumor site (ICD-9 code), which is consistent with previous reports.35, 36, 37 Patients with laryngeal tumors were at a lower risk of weight loss compared with patients with pharyngeal tumors. This was consistent with the univariate analysis (Table 1; tumor site, P = .04) and suggests that the propulsive swallowing structures may be more important than the coordinating role of the larynx in maintaining weight. Among the pharyngeal cancer patients, the next partition was the radiation dose to the masticatory muscles, which are responsible for chewing, and again underscore the effect of irradiating muscles involved in the swallow process. Patients with low doses to the masticatory muscles were further partitioned by the dose to the superior constrictor muscle, which was the most superiorly located muscle of the 3 pharyngeal constrictors that moved boluses downward into the esophagus. Patients who received >14 Gy to the masticatory muscles were further split by D78 larynx. For patients who received a larynx dose of ≥24 Gy, parotid D89 and age (ie, greater or less than 15 Gy and 58 years of age) were the last determinants of significant weight loss; parotid D89 and age affect swallow and parotid function, leading to weight changes. These findings were consistent with previous studies suggesting xerostomia was associated with a threshold of 26 Gy; late dysphagia was associated with a mean dose to pharyngeal constrictors and larynx ≤50 Gy.

The prediction tree at EOT (model 2) included the additional clinical assessment variables during treatment (Fig 4). The model demonstrated the following significant factors: (1) QoL: patient-reported oral intake; (2) dosimetry: dose to larynx and parotid; (3) RT toxicity: skin, nausea, and pain; and (4) shape relationship: the minimum distance between PTV and larynx. Compared with model 1 at RT planning, the additional clinical assessment variables improved the prediction of weight loss, as would be expected (Table 2, P = .03). AUC, sensitivity, specificity, and positive predictive and negative predictive values by cross-validation were 0.82, 0.98, 0.59, 0.46, and 0.99, respectively.

Figure 4

Prediction tree at the end of treatment (model 2) for critical WL among head and neck cancer patients 3 months posttreatment. Nodes display sample size (n) and the percentage of patients with critical weight loss within the (sub)group. The potential predictors in the CART analysis were patient reported outcome of oral intake, ICD-9 code, N stage, nausea, skin toxicity, pain intensity, dose to larynx and parotid, and low-dose PTV–larynx distance. PTV, planning target volume; other abbreviations as in Figure 3.

Model 2 demonstrated improved AUC (Table 2) at EOT from the incorporation of additional on-treatment clinical assessments (Figure 1, Figure 4). The most predictive factor was the patient-reported outcome “I am able to eat the food that I like?” in the Functional Assessment of Cancer Therapy-Head and Neck QoL instrument. It likely reflects the impact of treatment complications on oral intake, such as difficulty with chewing and swallowing as well as changes in taste, smell, and secretions.39, 40 Weight loss was observed if patients experienced a limited diet level at EOT and higher doses to larynx (D10 ≥42 Gy). If the patient enjoyed eating, the tumor site influenced the rest of the decision tree (larynx vs pharynx), which is consistent with model 1. Patients with larynx cancer experienced weight loss if they had severe acute skin and nausea toxicities (which we hypothesized were surrogate measures of tumor size and CRT) and high N-stage.

Discussion

Centered on the principle of PoC data capture about patients and their outcomes, we believe the potential to generate large datasets that can be exploited by the use of machine-learning algorithms, and with each successive patient, this can form the basis of a learning health system. As a pilot study, we used weight loss as our outcome measure because it is clinically important, requires intervention, and is accurately and consistently measured. To identify the highly multifactorial relationships between weight loss and related variables, we applied an exploratory approach to discover the important predictors for weight loss in our study population. This hypothesis-generating approach has the strength of prospective PoC data collection that is less prone to recall bias. Generating a large volume of heterogeneous and potentially more generalizable data can identify a spectrum of relationships, offering the potential to develop accurate personalized outcome prediction models. This analysis has demonstrated the potential importance of dosimetry and PROs in identifying modifiable weight loss risk factors. To demonstrate the adaptability of our prediction models and the importance of incorporating information during the treatment process, we developed 2 models at different time points during the care process of the HNC patient.

Breiman discussed the difference of the 2 cultures in statistical modeling (ie, statistics and machine learning). In traditional statistical modeling, we have an assumption of a data distribution first. The models with estimated parameters are validated particularly by goodness-of-fit tests. Machine learning, based on traditional statistics, allows more variables to be taken into account, and its algorithms suggest what is most important without those assumptions. Model validation is measured by its prediction accuracy particularly by cross-validation. Because of the complex nature in the causal relationship of weight loss, we pursued the latter machine learning approach (CART model) in this paper and evaluated its feasibility in knowledge discovery.

We used the CART algorithm as a knowledge–discovery tool because of its interpretability. The CART model provides critical variables threshold and their directional influence on the outcomes, for example, reducing the dose to the superior constrictor muscle will lead to no critical weight loss (Fig 3). In fact, it catalyzed the sharing of ideas between data scientists and clinicians to bridge the understanding across the multidisciplinary research group. CART can be used to explore datasets and can readily identify interactions among prognostic factors at each node in the tree. Unlike logistic regression approaches, CART does not require a specification of the function that is used to model covariates. These strengths are especially valuable in addressing the data heterogeneity commonly associated with clinical datasets. Collectively, our results provide a rich basis for future hypotheses regarding weight loss prevention in the irradiated HNC patient.

CART is particularly sensitive to missing data, however, which can remain a challenge even for prospectively collected datasets such as ours. Additional algorithms are available that can address the influence of missing data on the accuracy of a machine learning prediction. Despite this, the model performances generated in our feasibility study demonstrated AUC >0.75, which suggests promising accuracy warranting further investigations. Another challenge is model unstableness and overfitting when applying CART. To balance the recursive partitioning (branching) and pruning to overcome the potential instability in model development, we used leave-one-out cross-validation to address this issue. Although some other algorithms are known to have potentially high performance in prediction, we chose CART algorithm because of its interpretability.

Random forest is a tree-based algorithm that is known to be less prone to overfitting. Random forest grows many classification trees (eg, CART) by using randomly selected bootstrap samples and randomly selected predictor variable sets. Each tree gives a classification (vote) for a sample, and the forest chooses the classification having the most votes over all the trees in the forest. The steps of random predictor selection and voting give this algorithm high robustness in prediction, but the interpretability of the selected predictors is reduced to understanding which predictors carry high importance; the algorithm does not indicate the threshold and direction as the CART model does, making rationalization more difficult in the knowledge–discovery approach. Neural network is also subject to interpretability challenges; these methods are worthy of further exploration because their potential to improve prediction accuracy may outweigh their interpretability challenges.

Another possible concern is that the threshold cutoff for critical weight loss remains debated. Several studies have used a 5% or 10%18, 20, 21, 45, 46 threshold as a definition of critical weight loss based on the international consensus statement of the Academy of Nutrition and Dietetics and American Society for Parenteral and Enteral Nutrition. Our informatics platform with its data mining tools can readily be adapted to any threshold cutoff or definition of critical weight loss. In fact, we have built different CART weight loss prediction models using 5% as cutoff as an example (eFigs 1 and 2; available as supplementary material online only at www.practical.radonc.org, Table 1). Although they are not exactly the same tree models, we found common predictors (eFigs 1 and 2) that show robust associations with different weight-loss cutoffs that warrant attention in future big data approaches regarding weight loss prediction. In our analysis, we considered the absolute loss (≥5 kg) as the primary cutoff and used the relative loss (≥5%) as an auxiliary analysis. The commonly identified nodes are of particular interest because they may be important to weight loss regardless of cutoff. In summary, our flexible informatic infrastructure can readily facilitate building prediction models at multiple time points and cutoffs; more significantly, this lends itself to the iterative reanalysis of the models, especially by testing newly acquired factors that may be hypothesized to be relevant over time (Figure 1, Figure 2). This increases the probability of generating a valid, generalizable, and insightful model.

Last, our findings suggest that the informatics infrastructure that we have piloted can generate a large and diverse clinical dataset that can form the basis for advanced predictive modeling. We observed model 2 demonstrated that, with additional treatment toxicity information in the irradiated HNC patient, the accuracy of the prediction increased, suggesting that this approach has validity. The incorporation of more patient-specific information toward the end of RT improved the predictive accuracy of the weight loss model (AUC comparison between at planning and EOT: 5-kg model, P = .03; 5% model, P = .08) and is an encouraging indication that this infrastructure may facilitate the construction of a learning health system for precision radiation medicine. Our results were cross-validated by using a dataset in our institution. Demonstrating external validity as with traditional regression modeling approaches requires a comparable dataset to be available to then test the models that are generated. Further validation with other institutions' data needs to be investigated.

Conclusion

We have demonstrated the feasibility of creating 2 weight loss predictive models at different time points with improving receiver operating characteristics with incremental data. This reflects the potential of machine learning models in knowledge–discovery and potentially decision support from prospectively collected data in routine clinical workflow. Combined with large-scale analytic approaches, we believe that the Oncospace informatics framework can provide the foundation for developing a personalized learning health system, especially with the development of real-time clinical decision support systems.