Artificial Intelligence-Guided Prediction of Dental Doses Before Planning of Radiation Therapy for Oropharyngeal Cancer: Technical Development and Initial Feasibility of Implementation.

Purpose: The aim was to develop a novel artificial intelligence (AI)-guided clinical decision support system, to predict radiation doses to subsites of the mandible using diagnostic computed tomography scans acquired before any planning of head and neck radiation therapy (RT). Methods and Materials: A dose classifier was trained using RT plans from 86 patients with oropharyngeal cancer; the test set consisted of an additional 20 plans. The classifier was trained to predict whether mandible subsites would receive a mean dose >50 Gy. The AI predictions were prospectively evaluated and compared with those of a specialist head and neck radiation oncologist for 9 patients. Positive predictive value (PPV), negative predictive value (NPV), Pearson correlation coefficient, and Lin concordance correlation coefficient were calculated to compare the AI predictions to those of the physician.
Results: In the test data set, the AI predictions had a PPV of 0.95 and NPV of 0.88. For 9 patients evaluated prospectively, there was a strong correlation between the predictions of the AI algorithm and physician (P = .72, P < .001). Comparing the AI algorithm versus the physician, the PPVs were 0.82 versus 0.25, and the NPVs were 0.94 versus 1.0, respectively. Concordance between physician estimates and final planned doses was 0.62; this was 0.71 between AI-based estimates and final planned doses.
Conclusion: AI-guided decision support increased precision and accuracy of pre-RT dental dose estimates.

Introduction

Dental decay and osteoradionecrosis (ORN) are major long-term sequelae of head and neck radiation therapy (HN RT), but there is very little concrete basis to counsel patients in advance about their individual risk of potential complications.1, 2, 3 Dentists and oral surgeons often base their pre-RT assessments on the prescription doses that are anticipated by the consulting radiation oncologist but are rarely provided with actual dosimetry for specific subsites of the mandible. It is generally advised that in areas which are to receive a high dose of radiation (eg, >50 Gy), there should be a lifelong relative contraindication against dentoalveolar surgery, including procedures such as dental extractions, dental implants, periodontal surgery, and surgical endodontic therapy. Thus, prophylactic dental extractions are frequently performed before RT begins to remove any teeth that are felt to be compromised and which lie within a region anticipated to receive a high dose of radiation or teeth that are assessed to be nonrestorable by any means. Prophylactic dental extractions require weeks to heal and have potential to delay RT treatment by multiple weeks; the removal of teeth can also compromise patient speech and diet and potentially worsen their nutritional status just before beginning RT.

The utilization of data to drive improved RT treatment planning is a rapidly maturing field. In one application, a technique known as “knowledge-based planning” (KBP) uses historical planning information to very rapidly automate RT treatment planning of new cases. Machine-learning algorithms, such as logistic regression, linear models with penalty regularization, decision trees, random forest, gradient boosting, and deep learning approaches, have helped advance KBP., However, KBP approaches require full-scale contouring of clinical cases and are not able to provide anticipatory clinical decision support (CDS) before the initiation of contouring and RT planning processes of a specific case. Our proposed alternative CDS framework is intended to augment expert opinion before the point of contouring and/or planning and has been shown to help decision making at the point of care, such as at the initial consultation with the patient, while additionally avoiding some of the known limitations of artificial intelligence (AI), including lack of clinical context and unintended bias.

In this study, we developed an AI-based CDS tool to predict which subregions of the mandible would receive Dmean >50 Gy, before initiation of any actual RT planning activities. This CDS was based on a previously validated machine-learning algorithm that is capable of generating organ-specific dose estimates for new patients with oropharyngeal cancer based on historical data mining. A process was developed to predict doses to the mandible that would result from the eventual RT treatment plan based on minimal contouring of a diagnostic computed tomography (CT). The accuracy of this algorithm was compared with experience-based estimates provided by radiation oncologists specialized in the treatment of head and neck cancer. The CDS-based dose estimation was also evaluated prospectively on 9 patients, compared against the actual dosimetry that resulted from subsequent RT planning.

Methods and Materials

Machine learning model building and definition of mandible subsites on diagnostic CTs

The training data set, composed of 86 previously delivered RT treatment plans, including planning CT data sets with their associated structure sets and treatment plans with 3-dimensional dose distributions, was used for training this machine learning (ML) system (InsightRT v.2.4; Siris Medical, Burlingame, CA), which was customized for this application. Planning CTs were acquired with a Philips Vereos positron emission tomography/CT (512 × 512 resolution, 0.6-mm pixel spacing, 1.5-mm slice thickness). The training set also included associated diagnostic CTs, which were acquired with a GE Discovery STE (512 × 512 resolution, 1.27-mm pixel spacing, 3-mm slice thickness). Structure sets included complete target contour sets as well as subsite-based contouring of the mandible.

The 6 subsites of the mandible were manually contoured as separate structures for the purposes of training and testing. These were defined as follows: left posterior (teeth 17-19), left middle (teeth 20-22), left anterior (teeth 23-24), right anterior (teeth 25-26), right middle (teeth 27-29), and right posterior (teeth 30-32). The posterior subsites included the entire posterior mandible including the rami, coronoid processes, and condyloid processes.

The test data set, independent from the training data set, was created from 20 patients whose cases were chronologically subsequent to the training data set. The test set included not only planning CTs, structure sets, and the RT treatment plans, but also the associated diagnostic CTs for which the mandible was manually contoured for verification purposes. All of these patients were treated with curative intent, using intensity modulated radiation therapy prescribed to a total dose of at least 60 Gy.

Workflow for minimal contouring of diagnostic imaging

The workflow for contouring diagnostic imaging is shown in Fig. 1. Diagnostic CT scans were imported from the Picture and Archiving System (PACS; Agfa Healthcare, Mortsel, Belgium) into a commercial software (MiM Maestro v.6.9.3; MiM Software, Cleveland, OH) for manual delineation of target volumes. The primary and nodal gross tumor volumes (GTV) were delineated with neuroradiology input. A 5-mm expansion was added to the GTV to create the clinical target volume (CTV) and an additional 3-mm margin was added to create the planning tumor volume (PTV); an example is shown in Fig. 2. Elective nodal volumes were not delineated. The diagnostic CT associated with this minimal set of associated target contours (structure set) was then exported to the ML system.

Figure 1

Proposed clinical workflow of predicting dental dosimetry using a diagnostic CT and an artificial intelligence–based clinical decision support system. The contoured structures are listed in the middle box and the predicted doses to different regions of the mandible (A = anterior; P = posterior; R = right; L = left) are listed in the leftward box. This example shows red highlighted boxes that appear in the dose prediction results when preset quality parameters (in this case, mandible max, PL mean, 95% coverage of the PTV by prescribed dose) are not met. Abbreviations: CT = computed tomography; GTV = gross tumor volume; PTV = planning tumor volume.

Training and testing dose prediction to mandible subsites

A custom, autocontouring algorithm used the training data set to segment mandible subsites in the diagnostic CT scans (Fig. 2). First, a thresholding method (MiM Maestro) defined the mandible. Then, the following salient landmarks were identified in the CT and mandible (head tilt, lateral, anterior, and posterior extent of the mandible), and the mandible subsites were separated geometrically. Autocontouring accuracy was assessed using 2 methods. First, dosimetric error propagated by autocontouring was determined using the method proposed by Lim et al on the test data set; this method assesses contouring performance by comparing the final delivered dose volume (in units of Gray [Gy]) of the manually created structure to the dose-volume of the autocontoured structure. Second, autocontour accuracy on the submandible regions was evaluated using the Dice similarity coefficient, which is determined by dividing twice the volume of the intersection of the manual contour and the autocontour with the sum of the volume of manual contour and the volume of the autocontour.

Figure 2

Example of minimal required contouring on a diagnostic computed tomography scan for dose prediction: (A) autocontoured mandible subsites; (B) mandible subsites on the diagnostic computed tomography; (C) gross tumor volume (dark red inner contour) and planning tumor volume (lighter red outer contour) used for dose prediction as contoured on the diagnostic CT scan.

An ML dose prediction model was trained using a 5-fold cross-validation method; for each fold, 80% of the data were used to train the model, and 20% of the data (the validation data) were used to validate performance. Algorithmic performance was subsequently measured on the test data set. The dose prediction methodology has previously been described. Briefly, the model uses the regression ensemble boosting method with regularization (learning rate of 0.1), which forms a strong learner through iterative learning of weak learners, to solve for the mean doses to the subsite mandibular structures independently using features derived from the patient. These features were obtained from CT images, PTVs, organs at risk (OARs), and the oncology information system (Mosaiq; Elekta, Sweden). Features in this model incorporate prior assessment of Digital Imaging and Communications in Medicine images, medical record data (eg, International Classification of Diseases-9/10 diagnosis codes), radiation-transport parameters (eg, beam energy), and the treatment intent (eg, prescription dose). Of particular importance for this algorithm were the geometric features of the mandible substructures and the organs in close proximity to the mandible (eg, oral cavity); a table of feature importance, as determined by the features most represented in the boosted decision tree nodes, is given in Table 1.

Table 1

Top feature classes for mandible subsite dose prediction

Feature class	Description
1	Prescription to PTVs
2	Distance relationship between PTVs and mandible substructure
3	Volume of mandible substructures
4	Volume of PTVs
5	Projection of PTVs to mandible substructures
6	Geometric relationship of body to mandible substructures

Abbreviation: PTV = planning tumor volume.

The goal for this specific project was to predict the mean dose (Dmean) to each of the 6 mandible subsites and the overall maximum point dose (Dmax), defined as the dose to a point of 0.03 cc) to any part of the mandible. For the mandible subsites, the key dosimetric metric was Dmean >50 Gy, but Dmean >40 Gy and Dmean >30 Gy were also evaluated. Additionally, the dosimetric indices for the GTV, CTV, PTV, and OARs were analyzed and aggregated to form the treatment plan matrix.

The AI-predicted dosimetric indices were compared with the actual mandible dosimetry obtained after RT treatment planning. Positive predictive value (PPV) and negative predictive value (NPV) were used to evaluate the performance of the AI-based CDS in predicting Dmean of >50 Gy, >40 Gy, and >30 Gy to the 6 subsites of the mandible.

Prospective, correlative clinical comparison of CDS prediction against physician estimates and dosimetry from actual treatment plans

The diagnostic CT scans, each with an associated structure set containing the GTV and expanded CTVs and PTVs, were sent to the CDS. This was performed in advance of any treatment planning. Simultaneously, 2 head and neck specialist radiation oncologists provided estimates, based on their clinical experience, of 9 patients’ mandible subsite doses to the dental oncology team, as is our current institutional standard practice.

After the diagnostic CT scans had been sent to the CDS, the patients underwent standard CT-based simulation and RT treatment planning. Clinically accepted RT treatment plans were finalized using a commercial treatment planning software (Raystation; Raysearch Laboratories, Stockholm, Sweden); these were plans actually delivered to the patients. Using Pearson correlation and Lim concordance analysis, the accuracy of physician-estimated doses was compared with those predicted by the CDS, using the doses from accepted, deliverable treatment plans as the ground truth.

From these data, the PPV and NPV were used to evaluate the ability to predict whether each mandible subsite received Dmean >50 Gy, as shown in Table 2.

Table 2

Confusion matrix of estimated Dmean versus actual Dmean delivered to mandibular subsites

		Dmean actually delivered to mandible subsites
		Condition positive	Condition negative
Dmean estimated by either physician or CDS to mandibular subsites	Test outcome positive	TP total subsites correctly predicted and actually received Dmean >50 Gy	FP total subsites with overestimated dose prediction (did not receive Dmean >50 Gy)	PPVTP/(TP + FP)
	Test outcome negative	FN total subsites with underestimated dose prediction (received Dmean >50 Gy)	TN total subsites correctly predicted and actually did not receive Dmean >50 Gy	NPVTN/(FN + TN)

Abbreviations: CDS = clinical decision support; FN = false negative; FP = false positive; NPV = negative predictive value; PPV = positive predictive value; TN = true negative; TP = true positive.

The research was conducted under institutional review board–approved quantitative imaging analysis, No. 18-25441.

Results

Effect of autocontouring the mandible on the accuracy of dose prediction to mandible subsites in the test set

The computation of the dosimetric error propagated by mandible autocontouring (the error between the manual contours and the autocontours) demonstrated relatively small mean absolute errors (MAEs) in the prediction of mean doses to subsites of the mandible and the maximum dose to the entire mandible. In comparing results of autocontouring to manual contouring, the resulting average MAE was 1.5 Gy and all MAEs were within 3.30 Gy (Table 3). The average Dice coefficient for the mandible subsites was 81.5, where the anterior region was 72.4, the middle region was 83.6, and the posterior region was 88.3. The accuracy of predicting Dmean >50 Gy to the autocontoured mandible subsites on diagnostic CT scans in the test set resulted in a PPV of Dmean >50 Gy of 0.95 and an NPV of 0.88.

Table 3

Mean absolute errors in the validation test set with the use of autocontouring rather than manual contouring of the mandible and its subsites

Mandible subsite	MAE
Mandible LAM (Gy)	0.96
Mandible RAM (Gy)	1.05
Mandible LMM (Gy)	0.75
Mandible RMM (Gy)	1.38
Mandible LPM (Gy)	3.30
Mandible RPM (Gy)	2.93
Mandible max (Gy)	0.11

Abbreviations: LAM = anterior left mean; LMM = middle left mean; LPM = posterior left mean; MAE = mean absolute error; RAM = anterior right mean; RMM = middle right mean; RPM = posterior right mean.

Comparison of mandible dose prediction by CDS versus physician estimates

A strong correlation was observed between the AI-based CDS- and physician-generated predictions (Pearson correlation coefficient r = 0.72, P < .001). Using the delivered treatment plan as the ground truth, the PPVs for the CDS versus the physician were 0.82 versus 0.25, respectively, for predicting Dmean >50 Gy to the mandible subsites. The NPVs were 0.94 versus 1.0, respectively. Additional comparisons of PPVs and NPVs for several other dose thresholds (Dmean >40 Gy, Dmean >30 Gy) are shown in Table 4. The PPVs of the CDS predictions were equivalent or superior to the physicians’ estimates at every dose level. For further detail on predicting Dmean >50 Gy, the confusion matrix is shown in Table 5.

Table 4

NPVs and PPVs of predicting mean dose (Dmean) by the specialist radiation oncologist and AI-based clinical decision support system in 9 patients

	Specialist radiation oncologist		AI-based clinical decision support system
	NPV	PPV	NPV	PPV
Dmean >50 Gy	1.0	0.25	0.94	0.82
Dmean >40 Gy	0.97	0.62	0.89	0.95
Dmean >30 Gy	0.57	0.85	0.50	0.85

Abbreviations: AI = artificial intelligence; NPV = negative predictive value; PPV = positive predictive value.

Table 5

Confusion matrix for predicting Dmean >50 Gy for both the CDS and the physician

CDS		Actual
		Positive	Negative
Predicted	Positive	77.8%	3.1%
	Negative	4.8%	14.3%
Physician		Actual
		Positive	Negative
Predicted	Positive	77.7%	16.7%
	Negative	0%	5.6%

Abbreviation: CDS = clinical decision support.

We compared the ability to accurately predict mandible subsite dose >50 Gy between the CDS and the physician using the Wilcoxon rank test of equivalence (testing the null hypothesis that the CDS and the physician predictions are drawn from the same distribution). The accuracy of the CDS was statistically similar to the physician estimates (Wilcoxon rank test, P = .42). Although both the CDS and the physician were accurate for the anterior and posterior submandible structures (P < .01), when comparing the middle teeth, the CDS was statistically similar to the final delivered dose (P < .01), whereas the physician estimates were not (P = .4) (Wilcoxon rank test). Concordance between the physician's estimates and the delivered treatment plan doses was 0.62 and concordance between the CDS and the delivered treatment plan doses was 0.71.

Discussion

In this study, we established a novel AI CDS system that improves the accuracy and efficiency of pretreatment dental evaluation in patients scheduled for HN RT. The process was nested within our standard clinical workflow and concept-tested with the goal of efficiency and real-world clinical implementation. The CDS produced dose predictions to 6 subsites of the mandible that more closely approximated the actual treatment plan dosimetry than our current standard of care, which relies on physician estimates. The CDS only required GTV contouring on a diagnostic CT scan to generate these estimates.

There were multiple phases of this development process. First, we established that dose prediction accuracy could be maintained even if only an extremely limited feature set was contoured on a diagnostic CT scan and used for dose prediction; the only necessary contour was the GTV (auto-expanded to planning volumes). We considered whether it was more clinically useful to report maximum dose or mean dose to the mandible substructures. Ultimately, because the prediction to a point is less reliable, especially for such small structures, and less relevant clinically, we chose to focus on the mean dose to the mandible substructures. Second, based on these minimally contoured diagnostic CT scans, we evaluated the feasibility of dose prediction using the CDS to predict mean doses to 6 subsites of the mandible. Finally, we prospectively evaluated the ability of the CDS to predict whether specific subsites of the mandible would receive a mean dose >50 Gy. Of note, although our process was focused on the mandible, this same process could be generalized to predict doses to other organs of interest.

The CDS produced a similar NPV compared with a physician's estimate (0.94 and 1.0, respectively), but had a superior PPV of 0.82 compared with 0.25. Hence, the CDS was much less likely than the physician to overestimate high doses to the mandible, typically seen in the most posterior mandible. Thus, by reducing these overestimates, the CDS could prevent unneeded extraction procedures, reducing delays and potentially enabling more stable nutritional status before starting radiation. Furthermore, the CDS appeared to produce improved accuracy that was superior to the physicians’ estimates in the middle segments of the mandible, where lower radiation doses are typically delivered; improved accuracy to these lower-dose regions would allow for more nuanced and accurate preradiation therapy evaluation of teeth at an intermediate level of risk.

Qualified dental evaluation is an important preparatory step for all patients who are planning to undergo HN RT. The decision to perform extractions before RT is not entirely dependent on dental dosimetry, but in general, the expectation of a high radiation dose to an already compromised tooth will prompt a recommendation for prophylactic extraction. Extractions in the posterior mandible where high doses often occur usually involve molar teeth that are frequently deep-rooted or impacted; removal procedures are painful and sometimes risk lengthy or permanent nonhealing and/or alveolar nerve damage. After extractions, the initiation of RT may be delayed by a few to several weeks to allow for healing; this comes at the expense of delays in all patients and increasing the total treatment package time in postoperative patients, potentially compromising tumor control., Thus, by improving the specificity and accuracy of pre-RT dose estimation, our AI-based CDS has the potential to increase the overall efficiency of treatment.

The incidence of ORN after head and neck radiation treatment varies widely in the literature from 0.4% to up to 56%, and retrospective studies suggest an increased risk of ORN associated with higher doses of radiation to the mandible.18, 19, 20 Tsai et al reviewed 402 patients with oropharyngeal cancer and found that the incidence of ORN was 7.5% at a median time to development of 8 months. After adjustment for dental status (dentate vs postextraction), patients who developed ORN had larger volumes of the mandible exposed to high doses of radiation (>50 Gy). Another prospective study confirmed that >50 Gy to individual teeth increased the risk for ORN. Thus, a threshold of 50 Gy remains as a commonly used clinical reference point driving the decision on whether to enact pre-RT prophylactic extractions. The CDS in this project was more accurate than physicians in identifying situations above and below this standard threshold and would allow for more detailed discussion with patients about their individual level of risk.

One potential limitation of this CDS is the reliance on low-dimensional features that are delineated based on patient geometry. However, these types of features are often less affected by factors such as image noise and unintended correlations than other algorithms with higher dimensional features, such as those produced by neural nets. Additionally, from an efficiency standpoint, it was necessary to establish a process that could be implemented in the short window of time before actual RT treatment planning and which would not overburden physicians with excessive additional contouring. We were able to establish that by manually delineating only the GTV, the compromise in terms of dose-prediction accuracy was small, and in turn, a report could be produced in roughly 15 minutes from the start of contouring. Therefore, in a proposed clinical workflow, the only manual contouring truly required on the diagnostic CT scan would be the GTV, which would then be autoexpanded to the PTV, and the mandible subsites can be reliably autocontoured., This would likely be acceptable and feasible in the real-world setting of a busy clinical practice. Including more features, such as elective nodal volumes, likely would have resulted in more accurate prediction of mandibular doses below 50 Gy and would be an additional advancement of this work. Importantly, there is evidence supporting that doses below 50 Gy could contribute to ORN, and future refinements to the CDS should thus aim to improve the prediction of these lower mandibular doses. Finally, this CDS's application is currently limited to patients with oropharynx cancer and has yet to be tested for other head and neck cancer sites.

Conclusion

A novel AI-based CDS system can predict doses to high-risk OARs using only gross tumor volume contouring on a diagnostic CT obtained before initiation of actual radiation treatment planning. Development and implementation of AI-guided dose prediction would facilitate more precise estimations of dosimetrically-based risks to specific OARs.