Stefan Schandelmaier1, Matthias Briel2, Ravi Varadhan2, Christopher H Schmid2, Niveditha Devasenapathy2, Rodney A Hayward2, Joel Gagnier2, Michael Borenstein2, Geert J M G van der Heijden2, Issa J Dahabreh2, Xin Sun2, Willi Sauerbrei2, Michael Walsh2, John P A Ioannidis2, Lehana Thabane2, Gordon H Guyatt2. 1. Departments of Health Research Methods, Evidence, and Impact (Schandelmaier, Briel, Walsh, Thabane, Guyatt), Medicine (Walsh, Guyatt), Pediatrics (Thabane) and Anesthesia (Thabane), McMaster University, Hamilton, Ont.; Institute for Clinical Epidemiology and Biostatistics (Schandelmaier, Briel), Department of Clinical Research, Basel University, Basel, Switzerland; Division of Biostatistics and Bioinformatics (Varadhan), Sidney Kimmel Comprehensive Cancer Center, Johns Hopkins University, Baltimore, Md.; Department of Biostatistics (Schmid), Brown University School of Public Health, Brown University, Providence, RI; Indian institute of Public Health-Delhi (Devasenapathy), Public Health Foundation of India, New Delhi, India; VA Center for Clinical Management and Research (Hayward); Department of Internal Medicine (Hayward), University of Michigan School of Medicine; Department of Orthopaedic Surgery (Gagnier), University of Michigan; Department of Epidemiology (Gagnier), School of Public Health, University of Michigan, Ann Arbor, Mich.; Biostat (Borenstein), Englewood, NJ; Department of Social Dentistry (van der Heijden), Academic Center for Dentistry Amsterdam, University of Amsterdam and VU University Amsterdam, Amsterdam, Netherlands; Center for Evidence Synthesis in Health (Dahabreh) and Departments of Health Services, Policy, and Practice (Dahabreh) and Epidemiology (Dahabreh), School of Public Health, Brown University, Providence, RI; Chinese Evidence-Based Medicine Center (Sun), West China Hospital, Sichuan University, Chengdu, China; Institute of Medical Biometry and Statistics (Sauerbrei), Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany; Population Health Research Institute (Walsh), Hamilton Health Sciences/McMaster University, Hamilton, Ont.; Departments of Medicine (Ioannidis), Health Research and Policy (Ioannidis) and Biomedical Data Science (Ioannidis), and Statistics and Meta-Research Innovation Center at Stanford (METRICS) (Ioannidis), Stanford University, Stanford, Calif.; Biostatistics Unit (Thabane), St. Joseph's Healthcare, Hamilton, Ont. s.schandelmaier@gmail.com. 2. Departments of Health Research Methods, Evidence, and Impact (Schandelmaier, Briel, Walsh, Thabane, Guyatt), Medicine (Walsh, Guyatt), Pediatrics (Thabane) and Anesthesia (Thabane), McMaster University, Hamilton, Ont.; Institute for Clinical Epidemiology and Biostatistics (Schandelmaier, Briel), Department of Clinical Research, Basel University, Basel, Switzerland; Division of Biostatistics and Bioinformatics (Varadhan), Sidney Kimmel Comprehensive Cancer Center, Johns Hopkins University, Baltimore, Md.; Department of Biostatistics (Schmid), Brown University School of Public Health, Brown University, Providence, RI; Indian institute of Public Health-Delhi (Devasenapathy), Public Health Foundation of India, New Delhi, India; VA Center for Clinical Management and Research (Hayward); Department of Internal Medicine (Hayward), University of Michigan School of Medicine; Department of Orthopaedic Surgery (Gagnier), University of Michigan; Department of Epidemiology (Gagnier), School of Public Health, University of Michigan, Ann Arbor, Mich.; Biostat (Borenstein), Englewood, NJ; Department of Social Dentistry (van der Heijden), Academic Center for Dentistry Amsterdam, University of Amsterdam and VU University Amsterdam, Amsterdam, Netherlands; Center for Evidence Synthesis in Health (Dahabreh) and Departments of Health Services, Policy, and Practice (Dahabreh) and Epidemiology (Dahabreh), School of Public Health, Brown University, Providence, RI; Chinese Evidence-Based Medicine Center (Sun), West China Hospital, Sichuan University, Chengdu, China; Institute of Medical Biometry and Statistics (Sauerbrei), Faculty of Medicine and Medical Center, University of Freiburg, Freiburg, Germany; Population Health Research Institute (Walsh), Hamilton Health Sciences/McMaster University, Hamilton, Ont.; Departments of Medicine (Ioannidis), Health Research and Policy (Ioannidis) and Biomedical Data Science (Ioannidis), and Statistics and Meta-Research Innovation Center at Stanford (METRICS) (Ioannidis), Stanford University, Stanford, Calif.; Biostatistics Unit (Thabane), St. Joseph's Healthcare, Hamilton, Ont.
Abstract
BACKGROUND: Most randomized controlled trials (RCTs) and meta-analyses of RCTs examine effect modification (also called a subgroup effect or interaction), in which the effect of an intervention varies by another variable (e.g., age or disease severity). Assessing the credibility of an apparent effect modification presents challenges; therefore, we developed the Instrument for assessing the Credibility of Effect Modification Analyses (ICEMAN). METHODS: To develop ICEMAN, we established a detailed concept; identified candidate credibility considerations in a systematic survey of the literature; together with experts, performed a consensus study to identify key considerations and develop them into instrument items; and refined the instrument based on feedback from trial investigators, systematic review authors and journal editors, who applied drafts of ICEMAN to published claims of effect modification. RESULTS: The final instrument consists of a set of preliminary considerations, core questions (5 for RCTs, 8 for meta-analyses) with 4 response options, 1 optional item for additional considerations and a rating of credibility on a visual analogue scale ranging from very low to high. An accompanying manual provides rationales, detailed instructions and examples from the literature. Seventeen potential users tested ICEMAN; their suggestions improved the user-friendliness of the instrument. INTERPRETATION: The Instrument for assessing the Credibility of Effect Modification Analyses offers explicit guidance for investigators, systematic reviewers, journal editors and others considering making a claim of effect modification or interpreting a claim made by others.
BACKGROUND: Most randomized controlled trials (RCTs) and meta-analyses of RCTs examine effect modification (also called a subgroup effect or interaction), in which the effect of an intervention varies by another variable (e.g., age or disease severity). Assessing the credibility of an apparent effect modification presents challenges; therefore, we developed the Instrument for assessing the Credibility of Effect Modification Analyses (ICEMAN). METHODS: To develop ICEMAN, we established a detailed concept; identified candidate credibility considerations in a systematic survey of the literature; together with experts, performed a consensus study to identify key considerations and develop them into instrument items; and refined the instrument based on feedback from trial investigators, systematic review authors and journal editors, who applied drafts of ICEMAN to published claims of effect modification. RESULTS: The final instrument consists of a set of preliminary considerations, core questions (5 for RCTs, 8 for meta-analyses) with 4 response options, 1 optional item for additional considerations and a rating of credibility on a visual analogue scale ranging from very low to high. An accompanying manual provides rationales, detailed instructions and examples from the literature. Seventeen potential users tested ICEMAN; their suggestions improved the user-friendliness of the instrument. INTERPRETATION: The Instrument for assessing the Credibility of Effect Modification Analyses offers explicit guidance for investigators, systematic reviewers, journal editors and others considering making a claim of effect modification or interpreting a claim made by others.
Authors: Laura Koopman; Geert J M G van der Heijden; Paul P Glasziou; Diederick E Grobbee; Maroeska M Rovers Journal: J Clin Epidemiol Date: 2007-08-01 Impact factor: 6.437
Authors: Pablo Alonso-Coello; Holger J Schünemann; Jenny Moberg; Romina Brignardello-Petersen; Elie A Akl; Marina Davoli; Shaun Treweek; Reem A Mustafa; Gabriel Rada; Sarah Rosenbaum; Angela Morelli; Gordon H Guyatt; Andrew D Oxman Journal: BMJ Date: 2016-06-28
Authors: A Amery; W Birkenhäger; P Brixko; C Bulpitt; D Clement; P de Leeuw; J F de Plaen; M Deruyttere; A De Schaepdryver; C Dollery Journal: J Hypertens Suppl Date: 1986-12
Authors: Andrew J E Seely; Bram Rochwerg; Shannon M Fernando; Alexandre Tran; Behnam Sadeghirad; Karen E A Burns; Eddy Fan; Daniel Brodie; Laveena Munshi; Ewan C Goligher; Deborah J Cook; Robert A Fowler; Margaret S Herridge; Pierre Cardinal; Samir Jaber; Morten Hylander Møller; Arnaud W Thille; Niall D Ferguson; Arthur S Slutsky; Laurent J Brochard Journal: Intensive Care Med Date: 2021-11-25 Impact factor: 17.440
Authors: Jennie Johnstone; Maureen Meade; François Lauzier; John Marshall; Erick Duan; Joanna Dionne; Yaseen M Arabi; Diane Heels-Ansdell; Lehana Thabane; Daphnee Lamarche; Michael Surette; Nicole Zytaruk; Sangeeta Mehta; Peter Dodek; Lauralyn McIntyre; Shane English; Bram Rochwerg; Tim Karachi; William Henderson; Gordon Wood; Daniel Ovakim; Margaret Herridge; John Granton; M Elizabeth Wilcox; Alberto Goffi; Henry T Stelfox; Daniel Niven; John Muscedere; François Lamontagne; Frédérick D'Aragon; Charles St-Arnaud; Ian Ball; Dave Nagpal; Martin Girard; Pierre Aslanian; Emmanuel Charbonney; David Williamson; Wendy Sligl; Jan Friedrich; Neill K Adhikari; François Marquis; Patrick Archambault; Kosar Khwaja; Arnold Kristof; James Kutsogiannis; Ryan Zarychanski; Bojan Paunovic; Brenda Reeve; François Lellouche; Paul Hosek; Jennifer Tsang; Alexandra Binnie; Sébastien Trop; Osama Loubani; Richard Hall; Robert Cirone; Steve Reynolds; Paul Lysecki; Eyal Golan; Rodrigo Cartin-Ceba; Robert Taylor; Deborah Cook Journal: JAMA Date: 2021-09-21 Impact factor: 56.272
Authors: Martin L Blakely; Matthew A Rysavy; Kevin P Lally; Barry Eggleston; Claudia Pedroza; Jon E Tyson Journal: Semin Perinatol Date: 2022-06-10 Impact factor: 3.311
Authors: Martin L Blakely; Jon E Tyson; Kevin P Lally; Susan R Hintz; Barry Eggleston; David K Stevenson; Gail E Besner; Abhik Das; Robin K Ohls; William E Truog; Leif D Nelin; Brenda B Poindexter; Claudia Pedroza; Michele C Walsh; Barbara J Stoll; Rachel Geller; Kathleen A Kennedy; Reed A Dimmitt; Waldemar A Carlo; C Michael Cotten; Abbot R Laptook; Krisa P Van Meurs; Kara L Calkins; Gregory M Sokol; Pablo J Sanchez; Myra H Wyckoff; Ravi M Patel; Ivan D Frantz; Seetha Shankaran; Carl T D'Angio; Bradley A Yoder; Edward F Bell; Kristi L Watterberg; Colin A Martin; Carroll M Harmon; Henry Rice; Arlet G Kurkchubasche; Karl Sylvester; James C Y Dunn; Troy A Markel; Diana L Diesen; Amina M Bhatia; Alan Flake; Walter J Chwals; Rebeccah Brown; Kathryn D Bass; Shawn D St Peter; Christina M Shanti; Walter Pegoli; David Skarda; Joel Shilyansky; David G Lemon; Ricardo A Mosquera; Myriam Peralta-Carcelen; Ricki F Goldstein; Betty R Vohr; Isabell B Purdy; Abbey C Hines; Nathalie L Maitre; Roy J Heyne; Sara B DeMauro; Elisabeth C McGowan; Kimberly Yolton; Howard W Kilbride; Girija Natarajan; Kelley Yost; Sarah Winter; Tarah T Colaizy; Matthew M Laughon; Satyanarayana Lakshminrusimha; Rosemary D Higgins Journal: Ann Surg Date: 2021-10-01 Impact factor: 13.787
Authors: Zhikang Ye; Ying Wang; Long Ge; Gordon H Guyatt; David Collister; Waleed Alhazzani; Sean M Bagshaw; Emilie P Belley-Cote; Fang Fang; Liangying Hou; Philipp Kolb; Francois Lamontagne; Simon Oczkowski; Lonnie Pyne; Christian Rabbat; Matt Scaum; Borna Tadayon Najafabadi; Wimonchat Tangamornsuksan; Ron Wald; Qi Wang; Michael Walsh; Liang Yao; Linan Zeng; Abdullah Mohammed Algarni; Rachel J Couban; Paul Elias Alexander; Bram Rochwerg Journal: Crit Care Explor Date: 2021-05-12