Rectification of correlation by a sigmoid nonlinearity.

We investigated the normalized autocovariance (correlation coefficient) function of the output of an erf() function nonlinearity subject to non-zero mean Gaussian noise input. When the sigmoid is wide compared to the input, or the input mean is close to the midpoint of the sigmoid, the output correlation coefficient function is very close to the input correlation coefficient function. When the noise mean and variance are such that there is a significant probability of operating in the saturation region and the sigmoid is not too flat, the correlation coefficient of the output function is less than that of the input. This difference is much greater when the correlation coefficient is negative than when it is positive. The sigmoid partially rectifies the correlation coefficient function. The analysis does not depend on the spectral properties of the input noise. All that is required is that the input at times t and (t + tau) be jointly gaussian with the same mean and autocovariance. The analysis therefore applies equally well to the case of two identical sigmoids with jointly gaussian inputs. This correlational rectification could help explain the parameter sensitivity of "neural network" models. If biological neurons share this property it could explain why few negative correlations between spike trains have been observed.