Meghan McCullough1, Steven Ly1, Allyn Auslander1, Caroline Yao1, Alex Campbell1, Stefan Scherer1, William P Magee1. 1. From the Division of Plastic Surgery, Keck School of Medicine, and Department of Computer Science, University of Southern California; Division of Plastic Surgery, Children's Hospital of Los Angeles; Operation Smile; and Department of Plastic and Reconstructive Surgery, Shriners Hospital for Children.
Abstract
BACKGROUND: Despite the wide range of cleft lip morphology, consistent scales to categorize preoperative severity do not exist. Machine learning has been used to increase accuracy and efficiency in detection and rating of multiple conditions, yet it has not been applied to cleft disease. The authors tested a machine learning approach to automatically detect and measure facial landmarks and assign severity grades using preoperative photographs. METHODS: Preoperative images were collected from 800 unilateral cleft lip patients, manually annotated for cleft-specific landmarks, and rated using a previously validated severity scale by eight expert reviewers. Five convolutional neural network models were trained for landmark detection and severity grade assignment. Mean squared error loss and Pearson correlation coefficient for cleft width ratio, nostril width ratio, and severity grade assignment were calculated. RESULTS: All five models performed well in landmark detection and severity grade assignment, with the largest and most complex model, Residual Network, performing best (mean squared error, 24.41; cleft width ratio correlation, 0.943; nostril width ratio correlation, 0.879; severity correlation, 0.892). The mobile device-compatible network, MobileNet, also showed a high degree of accuracy (mean squared error, 36.66; cleft width ratio correlation, 0.901; nostril width ratio correlation, 0.705; severity correlation, 0.860). CONCLUSIONS: Machine learning models demonstrate the ability to accurately measure facial features and assign severity grades according to validated scales. Such models hold promise for the creation of a simple, automated approach to classifying cleft lip morphology. Further potential exists for a mobile telephone-based application to provide real-time feedback to improve clinical decision making and patient counseling.
BACKGROUND: Despite the wide range of cleft lip morphology, consistent scales to categorize preoperative severity do not exist. Machine learning has been used to increase accuracy and efficiency in detection and rating of multiple conditions, yet it has not been applied to cleft disease. The authors tested a machine learning approach to automatically detect and measure facial landmarks and assign severity grades using preoperative photographs. METHODS: Preoperative images were collected from 800 unilateral cleft lippatients, manually annotated for cleft-specific landmarks, and rated using a previously validated severity scale by eight expert reviewers. Five convolutional neural network models were trained for landmark detection and severity grade assignment. Mean squared error loss and Pearson correlation coefficient for cleft width ratio, nostril width ratio, and severity grade assignment were calculated. RESULTS: All five models performed well in landmark detection and severity grade assignment, with the largest and most complex model, Residual Network, performing best (mean squared error, 24.41; cleft width ratio correlation, 0.943; nostril width ratio correlation, 0.879; severity correlation, 0.892). The mobile device-compatible network, MobileNet, also showed a high degree of accuracy (mean squared error, 36.66; cleft width ratio correlation, 0.901; nostril width ratio correlation, 0.705; severity correlation, 0.860). CONCLUSIONS: Machine learning models demonstrate the ability to accurately measure facial features and assign severity grades according to validated scales. Such models hold promise for the creation of a simple, automated approach to classifying cleft lip morphology. Further potential exists for a mobile telephone-based application to provide real-time feedback to improve clinical decision making and patient counseling.
Authors: Abdulrahman Takiddin; Mohammad Shaqfeh; Osman Boyaci; Erchin Serpedin; Mitchell A Stotland Journal: Plast Reconstr Surg Glob Open Date: 2022-01-18