Steven J Holfinger1, M Melanie Lyons2, Brendan T Keenan3, Diego R Mazzotti4, Jesse Mindel2, Greg Maislin3, Peter A Cistulli5, Kate Sutherland5, Nigel McArdle6, Bhajan Singh6, Ning-Hung Chen7, Thorarinn Gislason8, Thomas Penzel9, Fang Han10, Qing Yun Li11, Richard Schwab3, Allan I Pack3, Ulysses J Magalang2. 1. Division of Pulmonary, Critical Care, and Sleep Medicine, The Ohio State University Wexner Medical Center, Columbus, OH. Electronic address: Steven.Holfinger@osumc.edu. 2. Division of Pulmonary, Critical Care, and Sleep Medicine, The Ohio State University Wexner Medical Center, Columbus, OH. 3. Division of Sleep Medicine, Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA. 4. Division of Medical Informatics, Department of Internal Medicine, University of Kansas Medical Center, Kansas City, KS. 5. Charles Perkins Centre, Faculty of Medicine and Health, University of Sydney, Sydney, NSW, Australia; Department of Respiratory and Sleep Medicine, Royal North Shore Hospital Sydney, Sydney, NSW, Australia. 6. West Australian Sleep Disorders Research Institute, Sir Charles Gairdner Hospital, Nedlands, WA, Australia; School of Human Sciences, University of Western Australia, Crawley, WA, Australia. 7. Division of Pulmonary, Critical Care Medicine and Sleep Medicine, Chang Gung Memorial Hospital, Taoyuan City, Taiwan. 8. Department of Sleep Medicine, Landspitali University Hospital, Reykjavik, Iceland; Medical Faculty, University of Iceland, Reykjavik, Iceland. 9. Interdisciplinary Center of Sleep Medicine, Charité University Hospital, Berlin, Germany. 10. Department of Respiratory Medicine, Peking University, Beijing, China. 11. Department of Respiratory and Critical Care Medicine, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China.
Abstract
BACKGROUND: Prediction tools without patient-reported symptoms could facilitate widespread identification of OSA. RESEARCH QUESTION: What is the diagnostic performance of OSA prediction tools derived from machine learning using readily available data without patient responses to questionnaires? Also, how do they compare with STOP-BANG, an OSA prediction tool, in clinical and community-based samples? STUDY DESIGN AND METHODS: Logistic regression and machine learning techniques, including artificial neural network (ANN), random forests (RF), and kernel support vector machine, were used to determine the ability of age, sex, BMI, and race to predict OSA status. A retrospective cohort of 17,448 subjects from sleep clinics within the international Sleep Apnea Global Interdisciplinary Consortium (SAGIC) were randomly split into training (n = 10,469) and validation (n = 6,979) sets. Model comparisons were performed by using the area under the receiver-operating curve (AUC). Trained models were compared with the STOP-BANG questionnaire in two prospective testing datasets: an independent clinic-based sample from SAGIC (n = 1,613) and a community-based sample from the Sleep Heart Health Study (n = 5,599). RESULTS: The AUCs (95% CI) of the machine learning models were significantly higher than logistic regression (0.61 [0.60-0.62]) in both the training and validation datasets (ANN, 0.68 [0.66-0.69]; RF, 0.68 [0.67-0.70]; and kernel support vector machine, 0.66 [0.65-0.67]). In the SAGIC testing sample, the ANN (0.70 [0.68-0.72]) and RF (0.70 [0.68-0.73]) models had AUCs similar to those of the STOP-BANG (0.71 [0.68-0.72]). In the Sleep Heart Health Study testing sample, the ANN (0.72 [0.71-0.74]) had AUCs similar to those of STOP-BANG (0.72 [0.70-0.73]). INTERPRETATION: OSA prediction tools using machine learning without patient-reported symptoms provide better diagnostic performance than logistic regression. In clinical and community-based samples, the symptomless ANN tool has diagnostic performance similar to that of a widely used prediction tool that includes patient symptoms. Machine learning-derived algorithms may have utility for widespread identification of OSA.
BACKGROUND: Prediction tools without patient-reported symptoms could facilitate widespread identification of OSA. RESEARCH QUESTION: What is the diagnostic performance of OSA prediction tools derived from machine learning using readily available data without patient responses to questionnaires? Also, how do they compare with STOP-BANG, an OSA prediction tool, in clinical and community-based samples? STUDY DESIGN AND METHODS: Logistic regression and machine learning techniques, including artificial neural network (ANN), random forests (RF), and kernel support vector machine, were used to determine the ability of age, sex, BMI, and race to predict OSA status. A retrospective cohort of 17,448 subjects from sleep clinics within the international Sleep Apnea Global Interdisciplinary Consortium (SAGIC) were randomly split into training (n = 10,469) and validation (n = 6,979) sets. Model comparisons were performed by using the area under the receiver-operating curve (AUC). Trained models were compared with the STOP-BANG questionnaire in two prospective testing datasets: an independent clinic-based sample from SAGIC (n = 1,613) and a community-based sample from the Sleep Heart Health Study (n = 5,599). RESULTS: The AUCs (95% CI) of the machine learning models were significantly higher than logistic regression (0.61 [0.60-0.62]) in both the training and validation datasets (ANN, 0.68 [0.66-0.69]; RF, 0.68 [0.67-0.70]; and kernel support vector machine, 0.66 [0.65-0.67]). In the SAGIC testing sample, the ANN (0.70 [0.68-0.72]) and RF (0.70 [0.68-0.73]) models had AUCs similar to those of the STOP-BANG (0.71 [0.68-0.72]). In the Sleep Heart Health Study testing sample, the ANN (0.72 [0.71-0.74]) had AUCs similar to those of STOP-BANG (0.72 [0.70-0.73]). INTERPRETATION: OSA prediction tools using machine learning without patient-reported symptoms provide better diagnostic performance than logistic regression. In clinical and community-based samples, the symptomless ANN tool has diagnostic performance similar to that of a widely used prediction tool that includes patient symptoms. Machine learning-derived algorithms may have utility for widespread identification of OSA.
Authors: Adam V Benjafield; Najib T Ayas; Peter R Eastwood; Raphael Heinzer; Mary S M Ip; Mary J Morrell; Carlos M Nunez; Sanjay R Patel; Thomas Penzel; Jean-Louis Pépin; Paul E Peppard; Sanjeev Sinha; Sergio Tufik; Kate Valentine; Atul Malhotra Journal: Lancet Respir Med Date: 2019-07-09 Impact factor: 30.700
Authors: S F Quan; B V Howard; C Iber; J P Kiley; F J Nieto; G T O'Connor; D M Rapoport; S Redline; J Robbins; J M Samet; P W Wahl Journal: Sleep Date: 1997-12 Impact factor: 5.849
Authors: Fabiola G Rizzatti; Diego R Mazzotti; Jesse Mindel; Greg Maislin; Brendan T Keenan; Lia Bittencourt; Ning-Hung Chen; Peter A Cistulli; Nigel McArdle; Frances M Pack; Bhajan Singh; Kate Sutherland; Bryndis Benediktsdottir; Ingo Fietze; Thorarinn Gislason; Diane C Lim; Thomas Penzel; Bernd Sanner; Fang Han; Qing Yun Li; Richard Schwab; Sergio Tufik; Allan I Pack; Ulysses J Magalang Journal: Chest Date: 2020-04-15 Impact factor: 10.262