K Strand1, L I Strand, H Flaatten. 1. Health Services Research Centre, Akershus University Hospital, Lørenskog, Norway. stkr@sus.no
Abstract
PURPOSE: In this study we evaluated and compared the interrater reliability of SAPS II and SAPS 3 in order to measure the consistency of performance among different raters. METHOD: Ten junior doctors working at two general ICUs were trained in the use of SAPS II and SAPS 3 using a 2.5-h training program. After training they scored 24 cases in both systems. Scores were analyzed using intraclass correlation coefficient (ICC) statistics. In order to identify variables with low reliability, subscores were analyzed using the ICC, and single-variables were compared to a template score using weighted kappa statistics. RESULTS: The ICC (95% CI) of the scores was 0.84 (0.74, 0.91) in SAPS II and 0.80 (0.68, 0.89) in SAPS 3, which is considered adequate for both systems. Mean mortality predictions among the raters had a range of 0.12 in SAPS II and 0.19 in SAPS 3. Administrative data including age had high reliability, whereas variables based on diagnostic information had only moderate reliability. Laboratory data had consistently higher reliability than variables based on the interpretation of charts. CONCLUSION: Both SAPS II and SAPS 3 have adequate interrater reliability, but the standardized mortality ratios are still likely to be influenced by the rater's scoring practice.
PURPOSE: In this study we evaluated and compared the interrater reliability of SAPS II and SAPS 3 in order to measure the consistency of performance among different raters. METHOD: Ten junior doctors working at two general ICUs were trained in the use of SAPS II and SAPS 3 using a 2.5-h training program. After training they scored 24 cases in both systems. Scores were analyzed using intraclass correlation coefficient (ICC) statistics. In order to identify variables with low reliability, subscores were analyzed using the ICC, and single-variables were compared to a template score using weighted kappa statistics. RESULTS: The ICC (95% CI) of the scores was 0.84 (0.74, 0.91) in SAPS II and 0.80 (0.68, 0.89) in SAPS 3, which is considered adequate for both systems. Mean mortality predictions among the raters had a range of 0.12 in SAPS II and 0.19 in SAPS 3. Administrative data including age had high reliability, whereas variables based on diagnostic information had only moderate reliability. Laboratory data had consistently higher reliability than variables based on the interpretation of charts. CONCLUSION: Both SAPS II and SAPS 3 have adequate interrater reliability, but the standardized mortality ratios are still likely to be influenced by the rater's scoring practice.
Authors: Philipp G H Metnitz; Rui P Moreno; Eduardo Almeida; Barbara Jordan; Peter Bauer; Ricardo Abizanda Campos; Gaetano Iapichino; David Edbrooke; Maurizia Capuzzo; Jean-Roger Le Gall Journal: Intensive Care Med Date: 2005-08-17 Impact factor: 17.440
Authors: Massimo Antonelli; Elie Azoulay; Marc Bonten; Jean Chastre; Giuseppe Citerio; Giorgio Conti; Daniel De Backer; Herwig Gerlach; Goran Hedenstierna; Michael Joannidis; Duncan Macrae; Jordi Mancebo; Salvatore M Maggiore; Alexandre Mebazaa; Jean-Charles Preiser; Jerôme Pugin; Jan Wernerman; Haibo Zhang Journal: Intensive Care Med Date: 2011-01-04 Impact factor: 17.440