A sparsity-based stochastic pooling mechanism for deep convolutional neural networks.

A novel sparsity-based stochastic pooling which integrates the advantages of max-pooling, average-pooling and stochastic pooling is introduced. The proposed pooling is designed to balance the advantages and disadvantages of max-pooling and average-pooling by using the degree of sparsity of activations and a control function to obtain an optimized representative feature value ranging from average value to maximum value of a pooling region. The optimized representative feature value is employed for probability weights assignment of activations in normal distribution. The proposed pooling also adopts weighted random sampling with a reservoir for the sampling process to preserve the advantages of stochastic pooling. This proposed pooling is evaluated on several standard datasets in deep learning framework to compare with various classic pooling methods. Experimental results show that it has good performance on improving recognition accuracy. The influence of changes to the feature parameter on recognition accuracy is also investigated.