Yun Ji Lee1, Jae Yong Lee1, Jae Hoon Cho2, Ji Ho Choi1. 1. Department of Otorhinolaryngology-Head and Neck Surgery, College of Medicine, Soonchunhyang University, Bucheon Hospital, Bucheon, Korea. 2. Department of Otorhinolaryngology-Head and Neck Surgery, College of Medicine, Konkuk University, Seoul, Korea.
Abstract
STUDY OBJECTIVES: We evaluated the interrater reliabilities of manual polysomnography sleep stage scoring. We included all studies that employed Rechtschaffen and Kales rules or American Academy of Sleep Medicine standards. We sought the overall degree of agreement and those for each stage. METHODS: The keywords were "Polysomnography (PSG)," "sleep staging," "Rechtschaffen and Kales (R&K)," "American Academy of Sleep Medicine (AASM)," "interrater (interscorer) reliability," and "Cohen's kappa." We searched PubMed, OVID Medline, EMBASE, the Cochrane library, KoreaMed, KISS, and the MedRIC. The exclusion criteria included automatic scoring and pediatric patients. We collected data on scorer histories, scoring rules, numbers of epochs scored, and the underlying diseases of the patients. RESULTS: A total of 101 publications were retrieved; 11 satisfied the selection criteria. The Cohen's kappa for manual, overall sleep scoring was 0.76, indicating substantial agreement (95% confidence interval, 0.71-0.81; P < .001). By sleep stage, the figures were 0.70, 0.24, 0.57, 0.57, and 0.69 for the W, N1, N2, N3, and R stages, respectively. The interrater reliabilities for stage N2 and N3 sleep were moderate, and that for stage N1 sleep was only fair. CONCLUSIONS: We conducted a meta-analysis to generalize the variation in manual scoring of polysomnography and provide reference data for automatic sleep stage scoring systems. The reliability of manual scorers of polysomnography sleep stages was substantial. However, for certain stages, the results were poor; validity requires improvement. CITATION: Lee YJ, Lee JY, Cho JH, Choi JH. Interrater reliability of sleep stage scoring: a meta-analysis. J Clin Sleep Med. 2022;18(1):193-202.
STUDY OBJECTIVES: We evaluated the interrater reliabilities of manual polysomnography sleep stage scoring. We included all studies that employed Rechtschaffen and Kales rules or American Academy of Sleep Medicine standards. We sought the overall degree of agreement and those for each stage. METHODS: The keywords were "Polysomnography (PSG)," "sleep staging," "Rechtschaffen and Kales (R&K)," "American Academy of Sleep Medicine (AASM)," "interrater (interscorer) reliability," and "Cohen's kappa." We searched PubMed, OVID Medline, EMBASE, the Cochrane library, KoreaMed, KISS, and the MedRIC. The exclusion criteria included automatic scoring and pediatric patients. We collected data on scorer histories, scoring rules, numbers of epochs scored, and the underlying diseases of the patients. RESULTS: A total of 101 publications were retrieved; 11 satisfied the selection criteria. The Cohen's kappa for manual, overall sleep scoring was 0.76, indicating substantial agreement (95% confidence interval, 0.71-0.81; P < .001). By sleep stage, the figures were 0.70, 0.24, 0.57, 0.57, and 0.69 for the W, N1, N2, N3, and R stages, respectively. The interrater reliabilities for stage N2 and N3 sleep were moderate, and that for stage N1 sleep was only fair. CONCLUSIONS: We conducted a meta-analysis to generalize the variation in manual scoring of polysomnography and provide reference data for automatic sleep stage scoring systems. The reliability of manual scorers of polysomnography sleep stages was substantial. However, for certain stages, the results were poor; validity requires improvement. CITATION: Lee YJ, Lee JY, Cho JH, Choi JH. Interrater reliability of sleep stage scoring: a meta-analysis. J Clin Sleep Med. 2022;18(1):193-202.
Authors: Warren R Ruehland; Fergal J O'Donoghue; Robert J Pierce; Andrew T Thornton; Parmjit Singh; Janet M Copland; Bronwyn Stevens; Peter D Rochford Journal: Sleep Date: 2011-01-01 Impact factor: 5.849
Authors: Xiaozhe Zhang; Xiaosong Dong; Jan W Kantelhardt; Jing Li; Long Zhao; Carmen Garcia; Martin Glos; Thomas Penzel; Fang Han Journal: Sleep Breath Date: 2014-05-07 Impact factor: 2.816
Authors: Madeleine Grigg-Damberger; David Gozal; Carole L Marcus; Stuart F Quan; Carol L Rosen; Ronald D Chervin; Merill Wise; Daniel L Picchietti; Stephan H Sheldon; Conrad Iber Journal: J Clin Sleep Med Date: 2007-03-15 Impact factor: 4.062
Authors: Heidi Danker-Hopfe; Peter Anderer; Josef Zeitlhofer; Marion Boeck; Hans Dorn; Georg Gruber; Esther Heller; Erna Loretz; Doris Moser; Silvia Parapatics; Bernd Saletu; Andrea Schmidt; Georg Dorffner Journal: J Sleep Res Date: 2009-03 Impact factor: 3.981
Authors: Alain Riveros-Rivera; Thomas Penzel; Hanns-Christian Gunga; Oliver Opatz; Friedemann Paul; Lars Klug; Michael Boschmann; Anja Mähler Journal: Front Physiol Date: 2022-05-24 Impact factor: 4.755