Abstract
Introduction:
In a recent manuscript, Eklund et al. (2016) reported inflated false positive rates in functional magnetic resonance imaging (fMRI) using common software packages. Here, we would like to draw attention to an important aspect that was not addressed in this publication. Specifically, we note that statistical inferences obtained using the Gauss random field approach depend heavily on a pre-processing parameter that was not included in the analysis performed by Eklund et al. (2016), namely the spatial resolution to which the data are resampled and interpolated during pre-processing. This resampling is needed to align the data to a common anatomical template. Eklund et al. used the common default setting of 2×2×2 mm³. In response to the paper by Eklund et al., Flandin and Friston (2016) used a different setting of this parameter, namely 3×3×3 mm³. Together with a more stringent initial cluster-forming threshold, they did not observe inflated false positive rates. However, a spatial resolution of 2×2×2 mm³ is the default value in two major software packages (SPM, FSL) and, hence, it is likely to be used for processing fMRI data by these packages. Moreover, in previous work, Friston and colleagues (Hopfinger et al., 2000) stated that resampling to 2×2×2 mm³ renders the analysis "more sensitive". It is, thus, unclear what a valid setting for this parameter should be. Therefore, it is of substantial relevance to systematically assess its influence on statistical inference.
Methods:
We analyzed 47 resting-state fMRI data sets, each acquired at a spatial resolution of 3×3×4 mm³ with 300 volumes. Using a strategy analogous to Eklund et al. (2016) we imposed various fake designs including block- and event-related types. Analysis was performed using SPM12. Normalization included a resampling with three different voxel sizes: 3×3×3 mm³, 2×2×2 mm³, and 1×1×1 mm³. Using a family-wise error (FWE) correction for multiple comparisons based on the Gauss Random Field approach, we first evaluated each data set separately. Thereafter, we performed a group-level inference in which all 47 data sets were pooled.
Results:
On the individual level, we found that with higher resampling resolutions, the FWE-corrected p-values decrease systematically so that more and more false positives occur. Table 1 shows coordinates and p-values of two different clusters obtained by statistical analysis of resting-state fMRI data using an arbitrary on/off-design with a block length of 20 s. As expected by our null hypothesis, we did not obtain any positive clusters with 3×3×3 mm³ resolution. However, with 2×2×2 mm³, the p-values are already smaller leading to a significant cluster. With a resolution of 1×1×1 mm³, both clusters are significant. Figure 1 shows the same effect using a fake event-related design with two experimental conditions. In the group analysis, we also observed that the FWE-corrected p-values decreased systematically with higher resampling resolutions. Noise clusters became significant when using a smaller voxel size after normalization.
Conclusions:
It appears that there is a systematic dependence of the false positive rate on the resampling parameter with smaller voxel sizes leading to smaller FWE-corrected p-values and hence more false positives. While some dependence on pre-processing parameters may be inevitable, a systematic dependence of this type is clearly worrisome, because researchers may be tempted to interpolate their data until the desired statistical significance level is reached. Statistical inference should certainly not depend in such a systematic way on a pre-processing parameter that can be set ad libitum. Clearly, this issue requires further in-depth analysis.
