Jennifer L Goldman1, Wen-Hung Chung2, Brian R Lee3, Chun-Bing Chen2, Chun-Wei Lu2, Wolfram Hoetzenecker4, Robert Micheletti5, Sally Usdin Yasuda6, David J Margolis5,7, Neil H Shear8, Jeffery P Struewing9, Munir Pirmohamed10. 1. Department of Pediatrics, Divisions of Pediatric Infectious Diseases & Clinical Pharmacology, Children's Mercy Hospitals & Clinic, 2401 Gillham Rd., Kansas City, MO, 64108, USA. jlgoldman@cmh.edu. 2. Department of Dermatology, Drug Hypersensitivity Clinical and Research Center, Chang Gung Memorial Hospital, Keelung and Linkou Branches, College of Medicine, Chang Gung University, Taipei, Taiwan. 3. Division of Health Services and Outcomes Research, Children's Mercy Hospitals & Clinics, Kansas City, USA. 4. Department of Dermatology, University Hospital Linz, Linz, Austria. 5. Department of Dermatology, University of Pennsylvania, Philadelphia, USA. 6. Food and Drug Administration, Silver Spring, USA. 7. Department of Biostatistics, Epidemiology and Informatics, University of Pennsylvania, Philadelphia, USA. 8. Division of Dermatology, Department of Medicine, Sunnybrook Health Sciences Center and University of Toronto, Toronto, Canada. 9. Division of Genomic Medicine, National Human Genome Research Institute, National Institutes of Health, Bethesda, USA. 10. Department of Molecular and Clinical Pharmacology, University of Liverpool, Liverpool, UK.
Abstract
PURPOSE: Establishment of causality between drug exposure and adverse drug reactions (ADR) is challenging even for serious ADRs such as Stevens-Johnson syndrome/toxic epidermal necrolysis (SJS/TEN). Several causality assessment tools (CAT) exist, but the reliability and validity of such tools is variable. The objective of this study was to compare the reliability and validity of existing ADR CATs on SJS/TEN cases. METHODS: Seven investigators completed three CAT (ALDEN, Naranjo, Liverpool) for 10 SJS/TEN cases. Each CAT categorized the causality of 30 potential drugs as definite/very probable, probable, possible, or doubtful/unlikely. An additional reviewer provided expert opinion by designating the implicated drug(s) for each case. A Kappa score was generated to compare CAT responses both by method (reliability of all 7 reviewers, by CATs) and by reviewer (reliability of the 3 CAT, by reviewer). A c statistic was calculated to assess validity. RESULTS: Inter-rater reliability by CAT was poor to fair: ALDEN 0.22, Naranjo 0.11, and Liverpool 0.12. Reliability was highest when causality classification was definite/very probable (0.16-0.41). Similarly, intra-rater reliability by reviewer was poor. When comparing the validity of the overall CAT to expert reviewer, area under the curve was highest for ALDEN (c statistic 0.65) as compared to Liverpool (0.55) or Naranjo (0.54). CONCLUSION: Available CAT have poor reliability and validity for drug-induced SJS/TEN. Due to the importance of determining ADR causality for research, industry, and regulatory purposes, development of an enhanced tool that can incorporate data from immunological testing and pharmacogenetic results may strengthen CAT usefulness and applicability for drug-induced SJS/TEN.
PURPOSE: Establishment of causality between drug exposure and adverse drug reactions (ADR) is challenging even for serious ADRs such as Stevens-Johnson syndrome/toxic epidermal necrolysis (SJS/TEN). Several causality assessment tools (CAT) exist, but the reliability and validity of such tools is variable. The objective of this study was to compare the reliability and validity of existing ADR CATs on SJS/TEN cases. METHODS: Seven investigators completed three CAT (ALDEN, Naranjo, Liverpool) for 10 SJS/TEN cases. Each CAT categorized the causality of 30 potential drugs as definite/very probable, probable, possible, or doubtful/unlikely. An additional reviewer provided expert opinion by designating the implicated drug(s) for each case. A Kappa score was generated to compare CAT responses both by method (reliability of all 7 reviewers, by CATs) and by reviewer (reliability of the 3 CAT, by reviewer). A c statistic was calculated to assess validity. RESULTS: Inter-rater reliability by CAT was poor to fair: ALDEN 0.22, Naranjo 0.11, and Liverpool 0.12. Reliability was highest when causality classification was definite/very probable (0.16-0.41). Similarly, intra-rater reliability by reviewer was poor. When comparing the validity of the overall CAT to expert reviewer, area under the curve was highest for ALDEN (c statistic 0.65) as compared to Liverpool (0.55) or Naranjo (0.54). CONCLUSION: Available CAT have poor reliability and validity for drug-induced SJS/TEN. Due to the importance of determining ADR causality for research, industry, and regulatory purposes, development of an enhanced tool that can incorporate data from immunological testing and pharmacogenetic results may strengthen CAT usefulness and applicability for drug-induced SJS/TEN.