Hunter Lee1, Sunna Ahmad2, Michael Frazier1, Mehmet Murat Dundar3, Hakan Turkkahraman4. 1. Department of Orthodontics and Oral Facial Genetics, Indiana University School of Dentistry, 1121 West Michigan Street, 46202, Indianapolis, IN, USA. 2. Indiana University School of Dentistry, Indianapolis, IN, USA. 3. Department of Computer and Information Science, School of Science, Indiana University Purdue University Indianapolis, Indianapolis, IN, USA. 4. Department of Orthodontics and Oral Facial Genetics, Indiana University School of Dentistry, 1121 West Michigan Street, 46202, Indianapolis, IN, USA. haturk@iu.edu.
Abstract
PURPOSE: The primary purpose of this study was to develop a new machine learning model for the surgery/non-surgery decision in class III patients and evaluate the validity and reliability of this model. METHODS: The sample consisted of 196 skeletal class III patients. All the cases were allocated randomly, 136 to the training set and the remaining 60 to the test set. Using the test set, the success rate of the artificial neural network model was estimated, along with a 95% confidence interval. To predict surgical cases, we trained a binary classifier using two different methods: random forest (RF) and logistic regression (LR). RESULTS: Both the RF and the LR model showed high separability when classifying each patient for surgical or non-surgical treatment. RF achieved an area under the curve (AUC) of 0.9395 on the test set. 95% confidence intervals were computed by bootstrap sampling as lower bound = 0.7908 and higher bound = 0.9799. On the other hand, LR achieved an AUC of 0.937 on the test set. 95% confidence intervals were computed by bootstrap sampling as lower bound = 0.8467 and higher bound = 0.9812. CONCLUSIONS: RF and LR machine learning models can be used to generate accurate and reliable algorithms to successfully classify patients up to 90%. The features selected by the algorithms coincide with the clinical features that we as clinicians weigh heavily when determining a treatment plan. This study further supports that overjet, Wits appraisal, lower incisor angulation, and Holdaway H angle can be used as strong predictors in assessing a patient's surgical needs.
PURPOSE: The primary purpose of this study was to develop a new machine learning model for the surgery/non-surgery decision in class III patients and evaluate the validity and reliability of this model. METHODS: The sample consisted of 196 skeletal class III patients. All the cases were allocated randomly, 136 to the training set and the remaining 60 to the test set. Using the test set, the success rate of the artificial neural network model was estimated, along with a 95% confidence interval. To predict surgical cases, we trained a binary classifier using two different methods: random forest (RF) and logistic regression (LR). RESULTS: Both the RF and the LR model showed high separability when classifying each patient for surgical or non-surgical treatment. RF achieved an area under the curve (AUC) of 0.9395 on the test set. 95% confidence intervals were computed by bootstrap sampling as lower bound = 0.7908 and higher bound = 0.9799. On the other hand, LR achieved an AUC of 0.937 on the test set. 95% confidence intervals were computed by bootstrap sampling as lower bound = 0.8467 and higher bound = 0.9812. CONCLUSIONS: RF and LR machine learning models can be used to generate accurate and reliable algorithms to successfully classify patients up to 90%. The features selected by the algorithms coincide with the clinical features that we as clinicians weigh heavily when determining a treatment plan. This study further supports that overjet, Wits appraisal, lower incisor angulation, and Holdaway H angle can be used as strong predictors in assessing a patient's surgical needs.