Tamas Spisak1. 1. Center for Translational Neuro- and Behavioral Sciences, Institute for Diagnostic and Interventional Radiology and Neuroradiology, Center University Hospital Essen, Essen, D-45147, Germany.
Abstract
BACKGROUND: The lack of nonparametric statistical tests for confounding bias significantly hampers the development of robust, valid, and generalizable predictive models in many fields of research. Here I propose the partial confounder test, which, for a given confounder variable, probes the null hypotheses of the model being unconfounded. RESULTS: The test provides a strict control for type I errors and high statistical power, even for nonnormally and nonlinearly dependent predictions, often seen in machine learning. Applying the proposed test on models trained on large-scale functional brain connectivity data (N= 1,865) (i) reveals previously unreported confounders and (ii) shows that state-of-the-art confound mitigation approaches may fail preventing confounder bias in several cases. CONCLUSIONS: The proposed test (implemented in the package mlconfound; https://mlconfound.readthedocs.io) can aid the assessment and improvement of the generalizability and validity of predictive models and, thereby, fosters the development of clinically useful machine learning biomarkers.
BACKGROUND: The lack of nonparametric statistical tests for confounding bias significantly hampers the development of robust, valid, and generalizable predictive models in many fields of research. Here I propose the partial confounder test, which, for a given confounder variable, probes the null hypotheses of the model being unconfounded. RESULTS: The test provides a strict control for type I errors and high statistical power, even for nonnormally and nonlinearly dependent predictions, often seen in machine learning. Applying the proposed test on models trained on large-scale functional brain connectivity data (N= 1,865) (i) reveals previously unreported confounders and (ii) shows that state-of-the-art confound mitigation approaches may fail preventing confounder bias in several cases. CONCLUSIONS: The proposed test (implemented in the package mlconfound; https://mlconfound.readthedocs.io) can aid the assessment and improvement of the generalizability and validity of predictive models and, thereby, fosters the development of clinically useful machine learning biomarkers.
Authors: Jonathan D Power; Anish Mitra; Timothy O Laumann; Abraham Z Snyder; Bradley L Schlaggar; Steven E Petersen Journal: Neuroimage Date: 2013-08-29 Impact factor: 6.556
Authors: Jean-Philippe Fortin; Nicholas Cullen; Yvette I Sheline; Warren D Taylor; Irem Aselcioglu; Philip A Cook; Phil Adams; Crystal Cooper; Maurizio Fava; Patrick J McGrath; Melvin McInnis; Mary L Phillips; Madhukar H Trivedi; Myrna M Weissman; Russell T Shinohara Journal: Neuroimage Date: 2017-11-17 Impact factor: 6.556
Authors: Abraham Nunes; Hugo G Schnack; Christopher R K Ching; Ingrid Agartz; Theophilus N Akudjedu; Martin Alda; Dag Alnæs; Silvia Alonso-Lana; Jochen Bauer; Bernhard T Baune; Erlend Bøen; Caterina Del Mar Bonnin; Geraldo F Busatto; Erick J Canales-Rodríguez; Dara M Cannon; Xavier Caseras; Tiffany M Chaim-Avancini; Udo Dannlowski; Ana M Díaz-Zuluaga; Bruno Dietsche; Nhat Trung Doan; Edouard Duchesnay; Torbjørn Elvsåshagen; Daniel Emden; Lisa T Eyler; Mar Fatjó-Vilas; Pauline Favre; Sonya F Foley; Janice M Fullerton; David C Glahn; Jose M Goikolea; Dominik Grotegerd; Tim Hahn; Chantal Henry; Derrek P Hibar; Josselin Houenou; Fleur M Howells; Neda Jahanshad; Tobias Kaufmann; Joanne Kenney; Tilo T J Kircher; Axel Krug; Trine V Lagerberg; Rhoshel K Lenroot; Carlos López-Jaramillo; Rodrigo Machado-Vieira; Ulrik F Malt; Colm McDonald; Philip B Mitchell; Benson Mwangi; Leila Nabulsi; Nils Opel; Bronwyn J Overs; Julian A Pineda-Zapata; Edith Pomarol-Clotet; Ronny Redlich; Gloria Roberts; Pedro G Rosa; Raymond Salvador; Theodore D Satterthwaite; Jair C Soares; Dan J Stein; Henk S Temmingh; Thomas Trappenberg; Anne Uhlmann; Neeltje E M van Haren; Eduard Vieta; Lars T Westlye; Daniel H Wolf; Dilara Yüksel; Marcus V Zanetti; Ole A Andreassen; Paul M Thompson; Tomas Hajek Journal: Mol Psychiatry Date: 2018-08-31 Impact factor: 15.992