Comment to: "Cluster failure revisited: Impact of first level design and physiological noise on cluster false positive rates".

The authors of the article entitled: “Cluster failure revisited: Impact of first level design and physiological noise on cluster false positive rates,” have further explored the cluster inference analytics and found out that using FSL FIX in conjunction with two‐sided testing would yield nominal results.

The authors state that Oulu data from the 1,000 Connectome database:”.…, the Oulu data should be avoided, because the assumption of no consistent activation over subjects is violated by the (physiological) noise,…”, since the FIX trained on 3 T data does not seem to clear the signal properly (Eklund et al., 2019). They also suggest that the 4 mm voxel size might be the cause for the problems, since physiological signal may become a dominant factor in the fMRI signal. However, as all the data was smoothed to 4 mm later on during preprocessing, this effect may not actually be the origin of the differences.

There seems to be a factor that the authors have missed here; the Oulu data collected around 2008 was scanned using 1.5 T GE Hdx scanner and not 3 T scanner like all the other datasets. This fact is not easy to detect from the NITRC download website and it might have been missed by the authors. At least 1.5 T is not mentioned in the aforementioned study at all. Moreover, there does not seem to be any comments on crucial imaging parameters such as flip angle, TE or TR from other analysed data. These parameters also have an effect on both signal and effect‐to‐noise ration of the BOLD data and can affect the FIX performance.

Furthermore, it is quite important to use FIX trained on similar fMRI data, as FIX uses ICA to detect noise and signal sources (Griffanti et al., 2014). ICA detects the inherent signal sources in a highly data driven way based on maximized non‐Gaussianity of the data (fMRI, EEG, MEG,…) density distributions (Kiviniemi, Kantola, Jauhiainen, Hyvärinen, & Tervonen, 2003). Therefore, the FIX algorithm needs to be trained on data that has similar noise characteristics, which in practice means the very same scanner from which the final fMRI signal is to be cleared from noise (Griffanti et al., 2014). After the FIX was trained on the 1.5 T Oulu data, as opposed to markedly different 3 T data, the authors actually do find nominal results, as the authors state”: Retraining the ICA FIX classifier finally lead to nominal results for the Oulu data.”

At this point, it is unclear why the Oulu 1.5 T data cannot be used, as it yields nominal results with correctly performed FIX. In our experience our good old 1.5 T data from Oulu is not better or worse than that of any other 1.5 T fMRI scanner data from that era. In the days of the data collection Oulu Functional NeuroImaging (http://www.oulu.fi/OFNI) group has published several ICA‐based papers that, for instance, enable functional clustering of the whole brain cortex into 42 RSN's (Kiviniemi et al., 2009 HBM Abou Elseoud et al., 2010, 2011). These results are virtually identical to RSN's found by Smith et al. at the same time simultaneously using 3 T data (Smith et al., 2009).

In conclusion, care should be taken when using fMRI datasets from shared datasets; technical properties such as scanner field strength or sampling rate need to be carefully identified and matched to suit for further analysis. If even one key issue of the data becomes unnoticed, both the analysis and the conclusions from the data can be completely misleading, even though the analysis is done with the best possible tools and high expertise.