Lorenzo Carvelli1, Alexander N Olesen2, Andreas Brink-Kjær2, Eileen B Leary3, Paul E Peppard4, Emmanuel Mignot3, Helge B D Sørensen5, Poul Jennum6. 1. Department of Health Technology, Technical University of Denmark, Kgs Lyngby, Denmark; Stanford University Center for Sleep Sciences and Medicine, Palo Alto, CA, USA. 2. Department of Health Technology, Technical University of Denmark, Kgs Lyngby, Denmark; Stanford University Center for Sleep Sciences and Medicine, Palo Alto, CA, USA; Danish Center for Sleep Medicine, Department of Clinical Neurophysiology, Rigshospitalet, Glostrup, Denmark. 3. Stanford University Center for Sleep Sciences and Medicine, Palo Alto, CA, USA. 4. Department of Population Health Sciences, University of Wisconsin-Madison, WI, USA. 5. Department of Health Technology, Technical University of Denmark, Kgs Lyngby, Denmark. 6. Danish Center for Sleep Medicine, Department of Clinical Neurophysiology, Rigshospitalet, Glostrup, Denmark. Electronic address: poul.joergen.jennum@regionh.dk.
Abstract
OBJECTIVE: Currently, manual scoring is the gold standard of leg movement scoring (LMs) and periodic LMs (PLMS) in overnight polysomnography (PSG) studies, which is subject to inter-scorer variability. The objective of this study is to design and validate an end-to-end deep learning system for the automatic scoring of LMs and PLMS in sleep. METHODS: The deep learning system was developed, validated and tested, with respect to manual annotations by expert technicians on 800 overnight PSGs using a leg electromyography channel. The study includes data from three cohorts, namely, the Wisconsin Sleep Cohort (WSC), Stanford Sleep Cohort (SSC) and MrOS Sleep Study. The performance of the system was further compared against individual expert technicians and existing PLM detectors. RESULTS: The system achieved an F1 score of 0.83, 0.71, and 0.77 for the WSC, SSC, and an ancillary study (Osteoporotic Fractures in Men Study, MrOS) cohorts, respectively. In a total of 60 PSGs from the WSC and the SSC scored by nine expert technicians, the system performed better than two and comparable to seven of the individual scorers with respect to a majority-voting consensus of the remaining scorers. In 60 PSGs from the WSC scored accurately for PLMS, the system outperformed four previous PLM detectors, which were all evaluated on the same data, with an F1 score of 0.85. CONCLUSIONS: The proposed system performs better or comparable to individual expert technicians while outperforming previous automatic detectors. Thereby, the study validates fully automatic methods for scoring LMs in sleep.
OBJECTIVE: Currently, manual scoring is the gold standard of leg movement scoring (LMs) and periodic LMs (PLMS) in overnight polysomnography (PSG) studies, which is subject to inter-scorer variability. The objective of this study is to design and validate an end-to-end deep learning system for the automatic scoring of LMs and PLMS in sleep. METHODS: The deep learning system was developed, validated and tested, with respect to manual annotations by expert technicians on 800 overnight PSGs using a leg electromyography channel. The study includes data from three cohorts, namely, the Wisconsin Sleep Cohort (WSC), Stanford Sleep Cohort (SSC) and MrOS Sleep Study. The performance of the system was further compared against individual expert technicians and existing PLM detectors. RESULTS: The system achieved an F1 score of 0.83, 0.71, and 0.77 for the WSC, SSC, and an ancillary study (Osteoporotic Fractures in Men Study, MrOS) cohorts, respectively. In a total of 60 PSGs from the WSC and the SSC scored by nine expert technicians, the system performed better than two and comparable to seven of the individual scorers with respect to a majority-voting consensus of the remaining scorers. In 60 PSGs from the WSC scored accurately for PLMS, the system outperformed four previous PLM detectors, which were all evaluated on the same data, with an F1 score of 0.85. CONCLUSIONS: The proposed system performs better or comparable to individual expert technicians while outperforming previous automatic detectors. Thereby, the study validates fully automatic methods for scoring LMs in sleep.
Keywords:
Automatic event detection; Leg movements during sleep; Manual scoring of polysomnography; Periodic leg movements during sleep; Polysomnography