Joint spectral-spatial information based classification is an active topic in hyperspectral remote sensing. Current classification approaches adopt a random sampling strategy to evaluate the performance of various classification systems. Due to the limitation of benchmark data, sampling of training and testing data is performed on the same image. In this paper, we point out that while training with random sampling is practical for hyperspectral image classification, it has intrinsic problems in evaluating spectral-spatial information based classifiers. This statement is supported by several experiments, and has lead to the proposal of a new sampling strategy for comparing spectral spatial information based classifiers.