Warren R Ruehland1,2, Peter D Rochford3,4, Robert J Pierce3,4, Parmjit Singh5, Andrew T Thornton5. 1. Institute for Breathing and Sleep, Studley Rd, Heidelberg, VIC, 3084, Australia. Warren.Ruehland@austin.org.au. 2. Austin Health, Heidelberg, VIC, Australia. Warren.Ruehland@austin.org.au. 3. Institute for Breathing and Sleep, Studley Rd, Heidelberg, VIC, 3084, Australia. 4. Austin Health, Heidelberg, VIC, Australia. 5. Royal Adelaide Hospital, Adelaide, South Australia, Australia.
Abstract
PURPOSE: This study evaluated whether or not polysomnography (PSG) inter-scorer reliability (ISR) across sleep centres could be improved by external proficiency testing (EPT), or by EPT combined with method alignment training. METHODS: Experienced scorers form 15 sleep centres were randomised to the following: (1) a control group, (2) a group that received a self-directed intervention of EPT reports (EPTPassive) or (3) a group that received an active intervention of method alignment training and EPT reports (EPTActive). Respiratory, arousal and sleep scoring ISR from sixteen PSG fragments were compared between groups across time. RESULTS: Among 30 scorers, there were no ISR changes in controls between baseline (BL) and 6 months (6 m). Both EPT groups showed ISR improvement from BL to 6 m for respiratory, arousal and sleep scoring (p < 0.05). Respiratory scoring back-transformed mean (95CI) proportion of specific agreement (PSA) for the EPTPassive group improved from 0.78 (0.72-0.84) to 0.80 (0.74-0.86) and for the EPTActive group from 0.80 (0.74-0.85) to 0.82 (0.76-0.88). Arousal scoring PSA for the EPTPassive group improved from 0.72 (0.66-0.77) to 0.74 (0.69-0.79) and for the EPTActive group from 0.71 (0.65-0.76) to 0.77 (0.72-0.82). Sleep scoring kappa for the EPTPassive group improved from 0.64 (0.58-0.69) to 0.73 (0.68-0.77) and for the EPTActive group from = 0.75 (0.71-0.80) to 0.80 (0.76-0.85). Overall, poorer performers achieved greater improvement. CONCLUSION: External proficiency testing produced modest, statistically significant PSG inter-scorer reliability improvements among experienced scorers across sleep centres, with potential to improve clinical management of individual patients and increase research study statistical power.
PURPOSE: This study evaluated whether or not polysomnography (PSG) inter-scorer reliability (ISR) across sleep centres could be improved by external proficiency testing (EPT), or by EPT combined with method alignment training. METHODS: Experienced scorers form 15 sleep centres were randomised to the following: (1) a control group, (2) a group that received a self-directed intervention of EPT reports (EPTPassive) or (3) a group that received an active intervention of method alignment training and EPT reports (EPTActive). Respiratory, arousal and sleep scoring ISR from sixteen PSG fragments were compared between groups across time. RESULTS: Among 30 scorers, there were no ISR changes in controls between baseline (BL) and 6 months (6 m). Both EPT groups showed ISR improvement from BL to 6 m for respiratory, arousal and sleep scoring (p < 0.05). Respiratory scoring back-transformed mean (95CI) proportion of specific agreement (PSA) for the EPTPassive group improved from 0.78 (0.72-0.84) to 0.80 (0.74-0.86) and for the EPTActive group from 0.80 (0.74-0.85) to 0.82 (0.76-0.88). Arousal scoring PSA for the EPTPassive group improved from 0.72 (0.66-0.77) to 0.74 (0.69-0.79) and for the EPTActive group from 0.71 (0.65-0.76) to 0.77 (0.72-0.82). Sleep scoring kappa for the EPTPassive group improved from 0.64 (0.58-0.69) to 0.73 (0.68-0.77) and for the EPTActive group from = 0.75 (0.71-0.80) to 0.80 (0.76-0.85). Overall, poorer performers achieved greater improvement. CONCLUSION: External proficiency testing produced modest, statistically significant PSG inter-scorer reliability improvements among experienced scorers across sleep centres, with potential to improve clinical management of individual patients and increase research study statistical power.
Authors: C W Whitney; D J Gottlieb; S Redline; R G Norman; R R Dodge; E Shahar; S Surovec; F J Nieto Journal: Sleep Date: 1998-11-01 Impact factor: 5.849
Authors: Warren R Ruehland; Thomas J Churchward; Linda M Schachter; Tristia Lakey; Natalie Tarquinio; Fergal J O'Donoghue; Maree Barnes; Peter D Rochford Journal: Sleep Med Date: 2014-11-26 Impact factor: 3.492
Authors: Ulysses J Magalang; Erna S Arnardottir; Ning-Hung Chen; Peter A Cistulli; Thorarinn Gíslason; Diane Lim; Thomas Penzel; Richard Schwab; Sergio Tufik; Allan I Pack Journal: J Clin Sleep Med Date: 2016-01 Impact factor: 4.062
Authors: Ulysses J Magalang; Ning-Hung Chen; Peter A Cistulli; Annette C Fedson; Thorarinn Gíslason; David Hillman; Thomas Penzel; Renaud Tamisier; Sergio Tufik; Gary Phillips; Allan I Pack Journal: Sleep Date: 2013-04-01 Impact factor: 5.849
Authors: Heidi Danker-Hopfe; Peter Anderer; Josef Zeitlhofer; Marion Boeck; Hans Dorn; Georg Gruber; Esther Heller; Erna Loretz; Doris Moser; Silvia Parapatics; Bernd Saletu; Andrea Schmidt; Georg Dorffner Journal: J Sleep Res Date: 2009-03 Impact factor: 3.981
Authors: Jingzhong Ding; F Javier Nieto; Norman J Beauchamp; Tamara B Harris; John A Robbins; Jacqueline B Hetmanski; Linda P Fried; Susan Redline Journal: Sleep Date: 2004-05-01 Impact factor: 5.849