Deep convolutional neural network has been used for pixel-wise hyperspectral image classification. However, convolutional operations only extract features from local neighborhood at a time, which is inefficient to capture long-range dependencies. On the other hand, the lack of training samples often leads to over-fitting problem. In this paper, we proposed a neural network which is formed by sequential local and non-local operation blocks. The proposed network takes hyperspectral image as input and outputs the class inference of each pixel. The local operation module extracts local spatial and spectral features. The non-local operation module computes the response at a position as a weighted sum of the features at all positions. So it can capture long-range dependencies without stacking deep layers. Experiments on two public datasets show that our proposed method outperforms several state-of-the-art methods using limited number of training samples.