Sayantanava Mitra1, Urvakhsh Meherwan Mehta2, Bhaskarapillai Binukumar3, Ganesan Venkatasubramanian2, Jagadisha Thirthalli2. 1. Central Queensland Mental Health, Alcohol and Other Drugs Services (CQMHAODS), Base Hospital, Rockhampton, Queensland, Australia; Faculty of Medicine, The University of Queensland Rural Clinical School, Rockhampton, Queensland, Australia. Electronic address: sayantanava@gmail.com. 2. Department of Psychiatry, National Institute of Mental Health and Neurosciences (NIMHANS), Bangalore, India. 3. Department of Biostatistics, National Institute of Mental Health and Neurosciences (NIMHANS), Bangalore, India.
Abstract
BACKGROUND: Non-invasive brain stimulation (NIBS) techniques have emerged as a promising tool for understanding and treating psychiatric disorders, necessitating a caution in terms of interpreting research results. OBJECTIVE: This study aimed at systematically evaluating a representative sample of research conducted using NIBS interventions in neuro-psychiatric conditions, and assessing the power these studies achieved, given their sample sizes. METHODS: A database search was conducted with defined keyword combinations. Using reported summary effects of the meta-analyses as estimate of the true effects, we calculated achieved power of each individual study to detect the effect indicated by the corresponding meta-analysis. RESULTS: Findings suggest that mean and median powers in the field of NIBS were 0.50, with a mode at 0.83 (range 0.05-1.00). When analysed separately, the median powers were 0.27 for tDCS, 0.70 for TMS and 0.97 for ECT. These studies had a mean total sample size of 22.2 ± 24.9 subjects and the median reported effect size across all studies was 0.61. CONCLUSION: According to our findings, studies conducted in NIBS miss around 50% of true positive results. Further, it appears that most of the researchers in this field chase statistical significance with small sample sizes, thus compromising the quality of their conclusions.
BACKGROUND: Non-invasive brain stimulation (NIBS) techniques have emerged as a promising tool for understanding and treating psychiatric disorders, necessitating a caution in terms of interpreting research results. OBJECTIVE: This study aimed at systematically evaluating a representative sample of research conducted using NIBS interventions in neuro-psychiatric conditions, and assessing the power these studies achieved, given their sample sizes. METHODS: A database search was conducted with defined keyword combinations. Using reported summary effects of the meta-analyses as estimate of the true effects, we calculated achieved power of each individual study to detect the effect indicated by the corresponding meta-analysis. RESULTS: Findings suggest that mean and median powers in the field of NIBS were 0.50, with a mode at 0.83 (range 0.05-1.00). When analysed separately, the median powers were 0.27 for tDCS, 0.70 for TMS and 0.97 for ECT. These studies had a mean total sample size of 22.2 ± 24.9 subjects and the median reported effect size across all studies was 0.61. CONCLUSION: According to our findings, studies conducted in NIBS miss around 50% of true positive results. Further, it appears that most of the researchers in this field chase statistical significance with small sample sizes, thus compromising the quality of their conclusions.
Authors: Daniel J Benjamin; James O Berger; Magnus Johannesson; Brian A Nosek; E-J Wagenmakers; Richard Berk; Kenneth A Bollen; Björn Brembs; Lawrence Brown; Colin Camerer; David Cesarini; Christopher D Chambers; Merlise Clyde; Thomas D Cook; Paul De Boeck; Zoltan Dienes; Anna Dreber; Kenny Easwaran; Charles Efferson; Ernst Fehr; Fiona Fidler; Andy P Field; Malcolm Forster; Edward I George; Richard Gonzalez; Steven Goodman; Edwin Green; Donald P Green; Anthony G Greenwald; Jarrod D Hadfield; Larry V Hedges; Leonhard Held; Teck Hua Ho; Herbert Hoijtink; Daniel J Hruschka; Kosuke Imai; Guido Imbens; John P A Ioannidis; Minjeong Jeon; James Holland Jones; Michael Kirchler; David Laibson; John List; Roderick Little; Arthur Lupia; Edouard Machery; Scott E Maxwell; Michael McCarthy; Don A Moore; Stephen L Morgan; Marcus Munafó; Shinichi Nakagawa; Brendan Nyhan; Timothy H Parker; Luis Pericchi; Marco Perugini; Jeff Rouder; Judith Rousseau; Victoria Savalei; Felix D Schönbrodt; Thomas Sellke; Betsy Sinclair; Dustin Tingley; Trisha Van Zandt; Simine Vazire; Duncan J Watts; Christopher Winship; Robert L Wolpert; Yu Xie; Cristobal Young; Jonathan Zinman; Valen E Johnson Journal: Nat Hum Behav Date: 2018-01
Authors: Leonardo M Knijnik; Jairo A Dussán-Sarria; Joanna R Rozisky; Iraci L S Torres; Andre R Brunoni; Felipe Fregni; Wolnei Caumo Journal: Pain Pract Date: 2015-01-12 Impact factor: 3.183
Authors: Jessica M Pisegna; Asako Kaneoka; William G Pearson; Sandeep Kumar; Susan E Langmore Journal: Clin Neurophysiol Date: 2015-05-09 Impact factor: 3.708
Authors: Estelle Dumas-Mallet; Katherine S Button; Thomas Boraud; Francois Gonon; Marcus R Munafò Journal: R Soc Open Sci Date: 2017-02-01 Impact factor: 2.963