BACKGROUND: The Naranjo criteria are frequently used for determination of causality for suspected adverse drug reactions (ADRs); however, the psychometric properties have not been studied in the critically ill. OBJECTIVE: To evaluate the reliability and validity of the Naranjo criteria for ADR determination in the intensive care unit (ICU). METHODS: All patients admitted to a surgical ICU during a 3-month period were enrolled. Four raters independently reviewed 142 suspected ADRs using the Naranjo criteria (review 1). Raters evaluated the 142 suspected ADRs 3-4 weeks later, again using the Naranjo criteria (review 2). Inter-rater reliability was tested using the kappa statistic. The weighted kappa statistic was calculated between reviews 1 and 2 for the intra-rater reliability of each rater. Cronbach alpha was computed to assess the inter-item consistency correlation. The Naranjo criteria were compared with expert opinion for criterion validity for each rater and reported as a Spearman rank (r(s)) coefficient. RESULTS: The kappa statistic ranged from 0.14 to 0.33, reflecting poor inter-rater agreement. The weighted kappa within raters was 0.5402-0.9371. The Cronbach alpha ranged from 0.443 to 0.660, which is considered moderate to good. The r(s) coefficient range was 0.385-0.545; all r(s) coefficients were statistically significant (p < 0.05). CONCLUSIONS: Inter-rater reliability is marginal; however, within-rater evaluation appears to be consistent. The inter-item correlation is expected to be higher since all questions pertain to ADRs. Overall, the Naranjo criteria need modification for use in the ICU to improve reliability, validity, and clinical usefulness.
BACKGROUND: The Naranjo criteria are frequently used for determination of causality for suspected adverse drug reactions (ADRs); however, the psychometric properties have not been studied in the critically ill. OBJECTIVE: To evaluate the reliability and validity of the Naranjo criteria for ADR determination in the intensive care unit (ICU). METHODS: All patients admitted to a surgical ICU during a 3-month period were enrolled. Four raters independently reviewed 142 suspected ADRs using the Naranjo criteria (review 1). Raters evaluated the 142 suspected ADRs 3-4 weeks later, again using the Naranjo criteria (review 2). Inter-rater reliability was tested using the kappa statistic. The weighted kappa statistic was calculated between reviews 1 and 2 for the intra-rater reliability of each rater. Cronbach alpha was computed to assess the inter-item consistency correlation. The Naranjo criteria were compared with expert opinion for criterion validity for each rater and reported as a Spearman rank (r(s)) coefficient. RESULTS: The kappa statistic ranged from 0.14 to 0.33, reflecting poor inter-rater agreement. The weighted kappa within raters was 0.5402-0.9371. The Cronbach alpha ranged from 0.443 to 0.660, which is considered moderate to good. The r(s) coefficient range was 0.385-0.545; all r(s) coefficients were statistically significant (p < 0.05). CONCLUSIONS: Inter-rater reliability is marginal; however, within-rater evaluation appears to be consistent. The inter-item correlation is expected to be higher since all questions pertain to ADRs. Overall, the Naranjo criteria need modification for use in the ICU to improve reliability, validity, and clinical usefulness.
Authors: Sandra L Kane-Gill; Shyam Visweswaran; Melissa I Saul; An-Kwok Ian Wong; Louis E Penrod; Steven M Handler Journal: Int J Med Inform Date: 2011-05-31 Impact factor: 4.046
Authors: Ruairi M Gallagher; Jamie J Kirkham; Jennifer R Mason; Kim A Bird; Paula R Williamson; Anthony J Nunn; Mark A Turner; Rosalind L Smyth; Munir Pirmohamed Journal: PLoS One Date: 2011-12-14 Impact factor: 3.240
Authors: Michael Knauer; Anton Haid; Karlheinz Ammann; Alois Lang; Felix Offner; Martina Türtscher; Peter Cerkl; Etienne Wenzl Journal: World J Surg Oncol Date: 2007-10-11 Impact factor: 2.754